Open Models Almanac
Methodsmodel architecturechannel mixermixture of experts

single mention · model architecture

Routed expert modulation

Multiplies the renormalized routed output by a constant from the config (2.5 in Laguna's). Laguna's report names it and points to DeepSeek-V3 and Nemotron 3 without stating the mechanism.

Written for the explainer The mixture-of-experts design space.

Also called Routed expert output modulation.

Filed under model architecture › channel mixer › mixture of experts