specific method · filed under model architecture
Granular Mixture of Experts
A mixture-of-experts design described as granular, distinguished from standard MoEs; the evidence does not specify its implementation.
Also called Granular MoEs.
- sources
- 2
- model
- 1
- labs adopt it
- 0
- strongest
- not used
How sources treat it
One count per evidence span, weakest treatment to strongest.
not used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
LatentMoE helped us achieve better accuracy per parameter than standard Granular MoEs
not usedmodel architecturein Nemotron 3 UltraNVIDIA
LatentMoE helped us achieve better accuracy per parameter than standard Granular MoEs
not usedmodel architecturein Nemotron 3 UltraNVIDIA
Filed alongside
Other methods under model architecture :: channel mixer :: mixture of experts :: fine-grained experts.