DeepSeek-V4.1-Flash
DeepSeekwe upgrade the originalmHC ... design to Single-PassmHC
1 more from this model
Additional architectural components include Single-Pass mHC (revised residual-stream mixing with an efficient Mega-mHC kernel)
specific method ยท filed under model architecture
mHC is a residual-connection method that maintains n residual streams between adjacent Transformer blocks.1 Each block updates the streams by mixing them with the block's output through token-wise coefficients predicted from the streams.2,3
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
maintains ๐ residual streams between adjacent Transformer blocksโ DeepSeek V41 Tech Report
๐๐+1 =๐ต ๐๐๐+๐ถ ๐F๐(๐ด๐๐๐โ DeepSeek V41 Tech Report
token-wise coefficients predicted from ๐๐โ DeepSeek V41 Tech Report
Also called Single-Pass mHC residual-stream mixing, Single-PassmHC, Single-PassmHC residual-stream mixing.
2 quotes from 2 sources, one card per model, each opening with its strongest.
we upgrade the originalmHC ... design to Single-PassmHC
Additional architectural components include Single-Pass mHC (revised residual-stream mixing with an efficient Mega-mHC kernel)