Open Models Almanac
Methodsmodel architecturechannel mixermixture of expertsshared experts

specific method · filed under model architecture

MoE without shared experts

A sparse mixture-of-experts feed-forward layer that has no shared experts.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See shared experts for the methods filed beside it.

Also called No shared experts, contains no shared experts, Sparse MoE FFNs without shared experts, without shared experts.

sources
3
models
3
lab adopt it
1
strongest
core

Documented in

Evidence

3 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
core 2not used 1
↑