specific method · filed under model architecture
Hyper-Connections
A residual-connectivity method that generalizes residual pathways using three learnable operators.
Also called Hyper-Connections (HC).
- sources
- 2
- models
- 2
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
evaluated 1core 1
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
The reference inference covers the vision encoder and aligner, DFlash attention, MoE, Hyper-Connections, and the DSpark forward path.
coremodel architecturein DeepSeek-V4-Flash-Vision-ExpDeepSeek
HC generalizes Eq. (21) and Eq. (22) into three learnable operators.
evaluatedunclearin Qwen3.8-Flash-NextQwen
Filed alongside
Other methods under model architecture :: normalization & residual.
Gated ResidualManifold-Constrained Hyper-ConnectionsAttention Residuals (AttnRes)GatedNormIdentity Hyper-ConnectionsRMSNormSingle-Pass mHCBlock AttnResIndependent Per-Branch NormalizationPer-Branch Scalar Write GateQK-ClipAll-branch Elementwise Read GateBounded Positive GatesData-Dependent Residual Read and Write OperatorsDynamic Gating of Residual Reads and WritesElementwise Data-Dependent Residual Read GateFour-Branch Residual StreamFour-Stream Hyper-Connection Combine UpdateFull Attention ResidualsLow-Rank Gated MixPost-Embedding RMSNormPre-LNSimplified AltUpSparse Gated Residual Writes