Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionsparse attention indexer

specific method · filed under model architecture

Lightning Indexer

The Lightning Indexer computes the index score between the query token and a preceding token, determining which tokens to be selected by the query token.1 This mechanism is used to retrieve only the key-value entries corresponding to the top-k index scores.2

Trade-off The indexer is computationally efficient because it has a small number of heads and can be implemented in FP8.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. The lightning indexer computes the index score between the query token and a preceding token , determining which tokens to be selected by the query token — DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
  2. Given the index scores for each query token , our fine-grained token selection mechanism retrieves only the key-value entries corresponding to the top-k index scores — DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
  3. Given that the lightning indexer has a small number of heads and can be implemented in FP8, its computational efficiency is remarkable — DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
sources
3
models
2
labs adopt it
2
strongest
core

Documented in

Z.ai (Zhipu AI)

Evidence

3 quotes from 3 sources, one card per model, each opening with its strongest.

How sources treat it
used 1core 2

DeepSeek-V3.2

DeepSeekadopted2 sources

The prototype of DSA primarily consists of two components: a lightning indexer and a fine-grained token selection mechanism.

coremodel architecturein DeepSeek-V3.2DeepSeek
1 more from this model

The prototype of DSA primarily consists of two components: a lightning indexer and a fine-grained token selection mechanism.

coremodel architecturein DeepSeek-V3.2DeepSeek

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑