Open Models Almanac
Methodsmodel architecturenormalization & residual

single mention · model architecture

Four-branch residual stream

Widens a single residual stream into four parallel branches.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See normalization & residual for the methods filed beside it.

Also called widens the residual stream into 4 branches.

Filed under model architecture › normalization & residual