single mention · model architecture
Gated DeltaNet MoE
An MoE architecture that combines gated delta networks with experts.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See mixture of experts for the methods filed beside it.
Also called gated-delta-networks MoE architecture.
Filed under model architecture › channel mixer › mixture of experts