mixture of experts

Model	filed here	expert routing	expert load balancing	shared experts	fine-grained experts	latent mixture of experts
GLM-5.3-Flash core	Mixture of Experts core—	Token-level expert routing core—	—	—	—	—
DeepSeek-V4.1-Flash core	Asymmetric input/output activation split coreMixture of Experts coreMoE with 384 routed experts and a shared expert coreMixture-of-Experts layers used—	—	Auxiliary-loss-free load balancing coreModality-specific auxiliary-loss-free load balancing coreformHC used—	Shared Experts used—	DeepSeekMoE shared and fine-grained routed experts core—	—
Hy4-preview core	Mixture of Experts coreMoE with routed and shared experts core—	Top-8 expert routing core—	—	Shared Experts coreTop-8 Routed-Expert Selection with Shared-Expert Activation core—	—	—
NVIDIA-Nemotron-3-Ultra-550B-A55B core	Mixture of Experts coreMixture-of-Experts layers core—	—	Expert Parallelism Load Balancing usedMaxVio evaluated—	Shared Experts used—	Granular Mixture of Experts not used—	Hybrid Latent Mixture of Experts coreLatentMoE core—
MiMo-V2.6-Flash core	Mixture of Experts core—	—	—	No Shared Experts core—	—	—
DeepSeek-V4-Flash core	DeepSeekMoE coreMixture of Experts coreMegaMoE used—	Hash routing usedAnticipatory Routing optionalLoss-spike-triggered Anticipatory Routing optional—	Auxiliary-loss-free load balancing used—	Shared Experts core—	—	—
MiMo-V2.5 core	Mixture of Experts coreMoE with 256 experts and top-8 routing used—	Top-8 expert routing used—	Expert bias update factor usedLoad balancing usedRound-robin load balancing optional—	No Shared Experts not used—	—	—
Hy3 core	Mixture of Experts core—	Routed experts coreTop-8 expert routing core—	—	Shared Experts core—	—	—
GLM-5.2 core	Mixture of Experts core—	DP-aware routing used—	—	—	—	—
MiniMax-M3 core	Mixture of Experts core—	Top-4 expert routing used—	Persistent load balancing used—	Shared Experts used—	—	—
DeepSeek-V3.2 core	—	Keep Routing core—	—	—	—	—
DeepSeek-V4-Flash-Vision-Exp core	Mixture of Experts core—	—	—	—	—	—
DeepSeek-V4-Pro core	DeepSeekMoE coreMixture of Experts coreMegaMoE used—	Anticipatory Routing usedHash routing usedLoss-spike-triggered Anticipatory Routing optional—	Auxiliary-loss-free load balancing used—	Shared Experts core—	—	—
Gemma 4 31B core	Mixture of Experts coreSparse expert activation core—	—	—	—	—	—
Inkling core	Mixture of Experts coreSigmoid-based MoE router with auxiliary-loss-free load balancing used—	Token-level expert routing coreTop-6 expert routing core—	—	MoE with Expert Sinks coreShared Experts Active on Every Token core—	—	—
Kimi K3 core	Dispatch recomputation coreMixture of Experts coreOutput-independent MoE gradient reformulation coreSparse expert activation core—	Fixed Top-k routing with frozen bias defaultLatent-space routing used—	Auxiliary-loss-free load balancing coreQuantile Balancing coreExact coordinate minimization usedExponential moving average of estimated quantiles usedHistogram-based quantile estimation usedPooled-global-batch quantile estimation used—	—	—	LatentMoE coreNormalized LatentMoE coreSharded latent weights with fused all-gather GEMM epilogue core—
Laguna-S-2.1 core	Mixture of Experts coreSparse expert activation coreRouted expert output modulation used—	Token-choice routing with softplus gating coreToken-choice routing used—	Auxiliary-loss load balancing used—	Shared Experts used—	—	—
MiMo-V2.5-Pro core	Mixture of Experts core—	—	—	—	—	—
MiMo-V2.6-Pro core	Mixture of Experts coreSparse expert activation core—	—	—	No Shared Experts core—	—	—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B core	Mixture of Experts coreMoE with 128 routed experts and a shared expert used—	Token-level expert routing core—	—	—	—	LatentMoE coreLatent-dimension reduction used—
Qwen3.5-397B-A17B core	Hybrid Mixture of Experts coreMixture of Experts coreSparse expert activation core—	—	—	10 Routed + 1 Shared Experts Activated per Token core8 Routed + 1 Shared Experts Activated per Token core—	—	—
Qwen3.6-35B-A3B core	Gated DeltaNet MoE coreMixture of Experts core—	—	—	—	—	—
Qwen3.8-Flash-Next core	Mixture of Experts coreFrequency-based partitioning of N-gram embedding slots evaluated—	—	—	10 Routed + 1 Shared Experts Activated per Token core—	—	—
Step-3.7-Flash core	Mixture of Experts core—	—	—	—	—	—
gpt-oss-120b core	Mixture of Experts core—	Top-k expert routing with softmax over selected experts core—	—	—	—	—

Model

filed here

expert routing

expert load balancing

shared experts

fine-grained experts

latent mixture of experts

GLM-5.3-Flash core

Mixture of Experts core—

Token-level expert routing core—

—

DeepSeek-V4.1-Flash core