Open Models Almanac
Methodsmodel architecturechannel mixermixture of expertsexpert load balancing

single source · model architecture

Modality-specific load balancing

Keeps a separate selection bias per modality, updated independently, so that balance is held among image tokens and among text tokens rather than only over their mix. DeepSeek-V4.1 reports it with no measurement of its effect.

Written for the explainer Expert load balancing.

Also called Modality-specific expert load balancing.

Filed under model architecture › channel mixer › mixture of experts › expert load balancing