Open Models Almanac
Methodsmodel architecturechannel mixermixture of experts

single mention · model architecture

Hybrid Mixture of Experts

A hybrid MoE backbone described as balancing capacity and efficiency across multimodal understanding and generation.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See mixture of experts for the methods filed beside it.

Also called Hybrid Attention Mixture-of-Experts (MoE).

Filed under model architecture › channel mixer › mixture of experts