single mention · model architecture
Routed expert modulation
Multiplies the renormalized routed output by a constant from the config (2.5 in Laguna's). Laguna's report names it and points to DeepSeek-V3 and Nemotron 3 without stating the mechanism.
Written for the explainer The mixture-of-experts design space.
Also called Routed expert output modulation.
Filed under model architecture › channel mixer › mixture of experts