Model techniques map
Taxonomymodel architecturechannel mixermixture of expertsshared experts

taxonomy node · level 4

shared experts

7 methods filed at this node or below it, from the sources of 14 models.

model architecture :: channel mixer :: mixture of experts :: shared experts

Matching aids for the classifier: shared expert; no shared experts.

In this branch 7

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 7

Shared Experts core · 7 sources · 7 quotes
No Shared Experts core · 3 sources · 3 quotes
Shared Experts Active on Every Token core · 2 sources · 2 quotes
MoE with Expert Sinks core · 1 source · 1 quote

By model

Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.