specific method · filed under model architecture
Fine-grained token selection mechanism
A fine-grained token selection mechanism is a sparse-attention component that, for each query token, selects which preceding tokens' key-value entries participate in attention.1 It retrieves only the key-value entries with the top-k index scores, computed by a lightning indexer, and applies attention between the query and those sparsely selected entries.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
determining which tokens to be selected by the query token
— paperretrieves only the key-value entries {c𝑠} corresponding to the top-k index scores
— paper