specific method · filed under model architecture
QK-Clip
A query-key clipping technique mentioned as a form of explicit activation control.
Also called QK-Clip technique, QK clipping for activation control.
- sources
- 2
- models
- 3
- labs adopt it
- 0
- strongest
- not used
How sources treat it
One count per evidence span, weakest treatment to strongest.
not used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
we do not employ the QK-Clip technique
not usedoptimizationin DeepSeek-V4DeepSeek
This allows stable training without explicit activation control such as qk-clip
not usedmodel architecturein Qwen3.8-Flash-NextQwen
Filed alongside
Other methods under model architecture :: normalization & residual.
Gated ResidualManifold-Constrained Hyper-ConnectionsAttention Residuals (AttnRes)GatedNormIdentity Hyper-ConnectionsRMSNormSingle-Pass mHCBlock AttnResHyper-ConnectionsIndependent Per-Branch NormalizationPer-Branch Scalar Write GateAll-branch Elementwise Read GateBounded Positive GatesData-Dependent Residual Read and Write OperatorsDynamic Gating of Residual Reads and WritesElementwise Data-Dependent Residual Read GateFour-Branch Residual StreamFour-Stream Hyper-Connection Combine UpdateFull Attention ResidualsLow-Rank Gated MixPost-Embedding RMSNormPre-LNSimplified AltUpSparse Gated Residual Writes