single mention · model architecture
Single-head index key
Single-head index key gives the sparse-attention index branch one shared key head that all heads use.1 Sharing the key makes the index-branch score computation nearly free while still indicating which KV blocks to examine.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
K_idx has only one head — all heads share the same index key
— MiniMax Goes Sparse: Decoding M3's Attention from a Single DiagramAs a result, Q_idx · K_idxᵀ costs almost nothing to compute
— MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram
Also called single-head key for index branch, single-head K_idx.
Filed under model architecture › token mixer › sparse attention › sparse attention indexer