methods · channel mixer
mixture of experts
76 methods filed at this node or below it, from the sources of 36 models.
In practice
3 features checked against each model’s own code and config.json, apart from its documents. A column’s page has the verdict and the lines.
adopteddiscussednot usedblank: documents silent✓in its code✕not in its code
| Model | Mixture of experts | Shared expert | Auxiliary-loss-free load balancing (selection bias) |
|---|---|---|---|
| MiniMax-M3.1-Flash | in its code | in its code | in its code |
| DeepSeek-V4.1-Flash-1017 | in its code | in its code | in its code |
| Step-5-Preview | in its code | in its code | in its code |
| Kolibri-1 | in its code | in its code | in its code |
| DeepSeek-V4.1-Flash | in its code | in its code | in its code |
| Hy4-preview | in its code | in its code | in its code |
| GLM-5.3-Flash | in its code | in its code | in its code |
| DeepSeek-V4-Flash-Vision-Exp | in its code | in its code | in its code |
| GLM-5.3 | in its code | in its code | in its code |
| DeepSeek-V4-Pro-0813 | in its code | in its code | in its code |
| NVIDIA-Nemotron-3.5-Lightning-30B-A3B | in its code | in its code | in its code |
| DeepSeek-V4-Flash-0731 | in its code | in its code | in its code |
| Laguna-S-2.1 | in its code | in its code | in its code |
| Kimi K3 | in its code | in its code | in its code |
| Inkling | in its code | in its code | in its code |
| Hy3 | in its code | in its code | in its code |
| GLM-5.2 | in its code | in its code | in its code |
| NVIDIA-Nemotron-3-Ultra-550B-A55B | in its code | in its code | in its code |
| MiniMax-M3 | in its code | in its code | in its code |
| Step-3.7-Flash | in its code | in its code | in its code |
| DeepSeek-V4-Flash | in its code | in its code | in its code |
| DeepSeek-V4-Pro | in its code | in its code | in its code |
| DeepSeek-V3.2 | in its code | in its code | in its code |
| Mistral-Large-4 | in its code | in its code | not in its code |
| NVIDIA-Nemotron-3.5-Super-VL-120B-A12B | in its code | in its code | code unclear |
| MiMo-V2.6-Flash | in its code | not in its code | in its code |
| MiMo-V2.6-Pro | in its code | not in its code | in its code |
| Qwen3.8-Flash-Next | in its code | in its code | not in its code |
| MiMo-V2.5 | in its code | not in its code | in its code |
| MiMo-V2.5-Pro | in its code | not in its code | in its code |
| Qwen3.6-35B-A3B | in its code | in its code | not in its code |
| Qwen3.5-397B-A17B | in its code | in its code | not in its code |
| gpt-oss-120b | in its code | not in its code | not in its code |
| LFM3-1.7B | not in its code | not in its code | not in its code |
| Xiaomi-Robotics-1 | not in its code | not in its code | not in its code |
| MolmoAct 2 | not in its code | not in its code | not in its code |
| Gemma 4 31B | not in its code | not in its code | not in its code |
| X-VLA | not in its code | not in its code | not in its code |
| OpenVLA / OpenVLA-OFT | not in its code | not in its code | not in its code |
In this branch 76
Everything filed at this node or below it, with one collapsible heading per child node.
filed here 17
12 from one source, or only mentioned as not used
expert routing 15
13 from one source, or only mentioned as not used
expert load balancing 25
20 from one source, or only mentioned as not used
By model
Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.