Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionsparse attention indexer

single mention · model architecture

IndexPool

Compresses groups of indexer key vectors through weighted pooling to reduce latency and memory use.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention indexer for the methods filed beside it.

Filed under model architecture › token mixer › sparse attention › sparse attention indexer