taxonomy node · level 4
expert load balancing
15 methods filed at this node or below it, from the sources of 8 models.
model architecture :: channel mixer :: mixture of experts :: expert load balancing
Matching aids for the classifier: auxiliary-loss-free load balancing; expert bias update factor; MaxVio; quantile balancing; round-robin balancing.
In this branch 15
Everything filed at this node or below it, with one collapsible heading per child node.
filed here 15
By model
Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.
| Model | techniques |
|---|---|
| DeepSeek-V4.1-Flash core | Auxiliary-loss-free load balancing coreModality-specific auxiliary-loss-free load balancing coreformHC used— |
| NVIDIA-Nemotron-3-Ultra-550B-A55B used | Expert Parallelism Load Balancing usedMaxVio evaluated— |
| DeepSeek-V4-Flash used | Auxiliary-loss-free load balancing used— |
| MiMo-V2.5 used | Expert bias update factor usedLoad balancing usedRound-robin load balancing optional— |
| MiniMax-M3 used | Persistent load balancing used— |
| DeepSeek-V4-Pro used | Auxiliary-loss-free load balancing used— |
| Kimi K3 core | Auxiliary-loss-free load balancing coreQuantile Balancing coreExact coordinate minimization usedExponential moving average of estimated quantiles usedHistogram-based quantile estimation usedPooled-global-batch quantile estimation used— |
| Laguna-S-2.1 used | Auxiliary-loss load balancing used— |