MiniMax-M3
MiniMaxselects the top- blocks via max-pooling scoring
1 more from this model
TopK decides which KV blocks to keep for this layer and this GQA group
implementation detail · filed under model architecture
Top-k block selection picks the highest-scoring KV blocks that a sparse attention layer will attend over.1,2 It ranks block-level scores and keeps only the top ones, so attention runs on the selected subset of the original K/V rather than the whole context.3
Summarised by glm-5.3-flash from 2 sources; every sentence is checked against the quote it cites.
TopK decides which KV blocks to keep for this layer— MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram
To efficiently select the top- KV blocks, the index module ranks the index scores directly— MiniMax Sparse Attention
Using I₁, I₂ from Step 1 as indices, we pull the corresponding block subsets out of the original K/V— MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram
Also called Top- block selection, top-k selection, TopK.
2 quotes from 2 sources, one card per model, each opening with its strongest.
selects the top- blocks via max-pooling scoring
TopK decides which KV blocks to keep for this layer and this GQA group