single mention · model architecture
Four-branch residual stream
Widens a single residual stream into four parallel branches.
The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See normalization & residual for the methods filed beside it.
Also called widens the residual stream into 4 branches.
Filed under model architecture › normalization & residual