single mention · model architecture
Dynamic gating of residual reads and writes
Gated Residual widens the residual stream into multiple branches in a model's residual connections.1 A dynamic gate controls reads from and writes to the residual stream, which strengthens cross-layer information flow and training stability.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
widens the residual stream into 4 branches
— Qwen3.8-Flash-Nextcontrols reads and writes with a dynamic gate, strengthening cross-layer information flow and training stability
— Qwen3.8-Flash-Next
Filed under model architecture › normalization & residual