single mention · model architecture
CSA2 compressor and indexer design
The described CSA2 design removes overlap and absolute positional embeddings and projects indexer keys from main KV entries.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention indexer for the methods filed beside it.
Filed under model architecture › token mixer › sparse attention › sparse attention indexer