Open Models Almanac
Methodsmodel architecturenormalization & residual

single mention · model architecture

Element-wise data-dependent residual read gate

A read gate modulates the information flowing through widened residual streams via an element-wise, data-dependent gate.1 It pairs the fine-grained read gate with a single scalar write gate per branch, keeping inference overhead low.2

Trade-off The gate gives finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. modulates information flowing through widened residual streams via an element-wise, data-dependent read gate — Qwen3.8-Flash-Next — Architecture, Evals & Inference Performance
  2. The read gate is element-wise and data-dependent; the write gate is a single scalar per branch — Qwen3.8-Flash-Next — Architecture, Evals & Inference Performance
  3. finer-grained expressiveness across layers while preserving training stability and keeping inference overhead low — Qwen3.8-Flash-Next — Architecture, Evals & Inference Performance

Also called read gate.

Filed under model architecture › normalization & residual