Open Models Almanac
Methodsmodel architecturetoken mixersparse attention

single mention · model architecture

Aggressive sparsity for inference efficiency

A sparsity increase described as improving M3.1 Flash inference efficiency, with no more specific mechanism identified.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention for the methods filed beside it.

Also called more aggressive sparsity.

Filed under model architecture › token mixer › sparse attention