specific method · filed under model architecture
Lightning Indexer
An indexer for sparse-attention token selection; one described implementation fuses score calculation, ReLU, and TopK into a single kernel.
- sources
- 3
- models
- 2
- labs adopt it
- 2
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models paper arxiv.orgintroduces DeepSeek Sparse Attention and the lightning indexer
- DeepSeek Sparse Attention explainer sebastianraschka.comRaschka's walkthrough of the lightning indexer and top-k selector
Evidence
3 spans quoted from the sources, strongest treatment first.
The prototype of DSA primarily consists of two components: a lightning indexer and a fine-grained token selection mechanism.
The prototype of DSA primarily consists of two components: a lightning indexer and a fine-grained token selection mechanism.
Lightning Indexer integrates score calculation, ReLU, and TopK operations into a single kernel
Filed alongside
Other methods under model architecture :: token mixer :: sparse attention :: sparse attention indexer.