single mention · model architecture
Sparse softmax attention
The general attention paradigm in which sparse attention is computed with softmax.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention for the methods filed beside it.
Also called sparse softmax attention paradigm.
Filed under model architecture › token mixer › sparse attention