Open Models Almanac
Methodsmodel architecturetoken mixersparse attention

single mention · model architecture

Two-stage sparse-attention formulation

A formulation described as having two stages, without further details on the stages or mechanism.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention for the methods filed beside it.

Also called two-stage sparse attention.

Filed under model architecture › token mixer › sparse attention