specific method · filed under model architecture
Manifold-Constrained Hyper-Connections
A Hyper-Connections variant that constrains residual mapping to the manifold of doubly stochastic matrices.
Also called mHC.
- sources
- 8
- models
- 4
- labs adopt it
- 2
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- mHC: Manifold-Constrained Hyper-Connections paper arxiv.org
- Manifold-Constrained Hyper-Connections (mHC) explainer sebastianraschka.com
Evidence
8 spans quoted from the sources, strongest treatment first.
The model adopts Manifold-Constrained Hyper-Connections to improve scaling efficiency.
The model also adopts Manifold-Constrained Hyper-Connections (mHC)
DeepSeek-V4 series incorporate Manifold-Constrained Hyper-Connections (mHC) to strengthen the conventional residual connections
Manifold-Constrained Hyper-Connections (mHC): constrains residual mapping onto the manifold of doubly stochastic matrices (Birkhoff polytope), enhancing signal propagation stability while preserving model expressivity.
mHC is confirmed in V4 as described in the December 2025 paper
incorporate mHC to strengthen conventional residual connections
Manifold-Constrained Hyper-Connections (mHC): We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity.
mHC (Xie et al., 2025) uses a sigmoid and additionally constrains to a manifold of doubly stochastic matrices.
Filed alongside
Other methods under model architecture :: normalization & residual.