Open Models Almanac
Methodsmodel architecturenormalization & residual

specific method ยท filed under model architecture

Single-Pass mHC

mHC is a residual-connection method that maintains n residual streams between adjacent Transformer blocks.1 Each block updates the streams by mixing them with the block's output through token-wise coefficients predicted from the streams.2,3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. maintains ๐‘› residual streams between adjacent Transformer blocks โ€” DeepSeek V41 Tech Report
  2. ๐‘‹๐‘™+1 =๐ต ๐‘™๐‘‹๐‘™+๐ถ ๐‘™F๐‘™(๐ด๐‘™๐‘‹๐‘™ โ€” DeepSeek V41 Tech Report
  3. token-wise coefficients predicted from ๐‘‹๐‘™ โ€” DeepSeek V41 Tech Report

Also called Single-Pass mHC residual-stream mixing, Single-PassmHC, Single-PassmHC residual-stream mixing.

sources
2
model
1
lab adopt it
1
strongest
core

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 1core 1
โ†‘