Qwen3.8-Flash-Next
Alibaba (Qwen)Gated Residual modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and a per-branch scalar write gate.
7 more from this model
Gated Residual (GR) widens the residual stream into 4 branches and controls reads and writes with a dynamic gate
the gate decides how that capacity is spent, while also supplying the rescaling that keeps training stable
GR supplies a rescaling that markedly improves training stability
We describe the concrete design of GR below.
The gated residual "modulates information flowing through widened residual streams via an element-wise, data-dependent read gate and per-branch scalar write gate"
The original single residual stream is expanded into 4 parallel branches, allowing the model to decide dynamically, based on the current content, how much information to read from each branch and how much to write back.
four residual branches dynamically control cross-layer reads and writes.