implementation detail · filed under model architecture
Top-4 expert routing
Selects the four highest-scoring experts for each token.
Also called top-4 expert selection, top-4 routed expert selection.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
Each MoE layer uses 128 routed experts, 1 shared expert, and top-4 routed expert selection.
usedmodel architecturein MiniMax-M3MiniMax
top-4 expert routing
usedmodel architecturein MiniMax Sparse AttentionMiniMax
Filed alongside
Other methods under model architecture :: channel mixer :: mixture of experts :: expert routing.
Top-8 expert routingToken-level expert routingAnticipatory RoutingDP-aware routingFixed Top-k routing with frozen biasHash routingKeep RoutingLatent-space routingLoss-spike-triggered Anticipatory RoutingRouted expertsToken-choice routingToken-choice routing with softplus gatingTop-6 expert routingTop-k expert routing with softmax over selected experts