single mention · model architecture
Hybrid sparse mixture-of-experts Transformer architecture
The sources on this page only mention it in passing. See hybrid layer stacking for the methods filed beside it.
Also called hybrid sparse MoE Transformer backbone.
Filed under model architecture › token mixer › hybrid layer stacking