Open Models Almanac
Methodsmodel architecturechannel mixermixture of expertsexpert routing

methods · mixture of experts

expert routing

15 methods filed at this node or below it, from the sources of 18 models.

In this branch 15

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 15

Top-k expert routing core · 11 sources · 12 quotes
Anticipatory Routing used · 2 sources · 2 quotes
13 from one source, or only mentioned as not used
Keep Routing core · 1 source · 1 quote
Sigmoid-based expert routing core · 1 source · 1 quote
Token-choice routing with softplus gating core · 1 source · 1 quote
Top-k routing with frozen bias default · 1 source · 1 quote
DP-aware routing used · 1 source · 1 quote
Hash routing used · 1 source · 1 quote
Latent-space routing used · 1 source · 1 quote
Token-choice routing used · 1 source · 1 quote
Loss-spike-triggered Anticipatory Routing optional · 1 source · 1 quote
Random routed-expert selection evaluated · 1 source · 1 quote
Heterogeneity-guided MoE routing not used · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

↑