Open Models Almanac
Methodsmodel architecturenormalization & residual

single mention · model architecture

Dynamic gating of residual reads and writes

Gated Residual widens the residual stream into multiple branches in a model's residual connections.1 A dynamic gate controls reads from and writes to the residual stream, which strengthens cross-layer information flow and training stability.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. widens the residual stream into 4 branches — Qwen3.8-Flash-Next
  2. controls reads and writes with a dynamic gate, strengthening cross-layer information flow and training stability — Qwen3.8-Flash-Next

Filed under model architecture › normalization & residual