specific method · filed under model architecture
LatentMoE
A mixture-of-experts architecture that routes experts operating in a compact latent space, separating routed-expert width from full model width.
Also called Latent Mixture of Experts, Latent Mixture-of-Experts (LatentMoE), Latent MoE, LatentMixture of Experts.
- sources
- 7
- models
- 3
- labs adopt it
- 2
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts (Elango et al., 2026) paper arxiv.org
- Latent MoE - LLM Architecture Gallery explainer sebastianraschka.com
Evidence
9 spans quoted from the sources, strongest treatment first.
LatentMoE [32] makes this expansion affordable by separating the full model width from the routed-expert width
LatentMoE (Elango et al., 2026) for MoE layers
LatentMoE (Elango et al., 2026), which buys more routed experts at fixed inference cost
The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture
The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture
LatentMoE helped us achieve better accuracy per parameter than standard Granular MoEs
LatentMoE (latentmoe_tr), which buys more routed experts at fixed inference cost
routed to an expanded set of experts that operate entirely in this latent space, and then projected back
Leverages LatentMoE for improved accuracy.
Filed alongside
Other methods under model architecture :: channel mixer :: mixture of experts :: latent mixture of experts.