single mention · model architecture
Element-wise data-dependent residual read gate
A read gate modulates the information flowing through widened residual streams via an element-wise, data-dependent gate.1 It pairs the fine-grained read gate with a single scalar write gate per branch, keeping inference overhead low.2
Trade-off The gate gives finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
modulates information flowing through widened residual streams via an element-wise, data-dependent read gate
— Qwen3.8-Flash-Next — Architecture, Evals & Inference PerformanceThe read gate is element-wise and data-dependent; the write gate is a single scalar per branch
— Qwen3.8-Flash-Next — Architecture, Evals & Inference Performancefiner-grained expressiveness across layers while preserving training stability and keeping inference overhead low
— Qwen3.8-Flash-Next — Architecture, Evals & Inference Performance
Also called read gate.
Filed under model architecture › normalization & residual