single mention · model architecture
Block-level selection
Block-level selection is a sparse-attention method in which each query selects a small number of KV blocks to attend to, instead of every token.1,2 Attention is then computed on the real (uncompressed) K/V, and queries in the same GQA group share one top-k selection so one set of KV blocks can be loaded in a single pass.3,4
Trade-off It keeps the full expressive power of softmax attention, but the KV cache doesn't shrink along with the attention sparsification.5
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
block-level selection like CSA but attention is computed on the real K/V
— MiniMax Goes Sparse: Decoding M3's Attention from a Single DiagramDuring decode, each query interacts only with the selected KV blocks
— MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagramblock-level selection, but attention computed on the real K/V
— MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagramquery heads within the same GQA group share a single top-k selection
— MiniMax Goes Sparse: Decoding M3's Attention from a Single DiagramThe cost is that the KV cache doesn't shrink along with attention sparsification
— MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram
Filed under model architecture › token mixer › sparse attention › block selection