Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionblock selection

implementation detail · filed under model architecture

Top-k block selection

Top-k block selection picks the highest-scoring KV blocks that a sparse attention layer will attend over.1,2 It ranks block-level scores and keeps only the top ones, so attention runs on the selected subset of the original K/V rather than the whole context.3

How this definition was written

Summarised by glm-5.3-flash from 2 sources; every sentence is checked against the quote it cites.

  1. TopK decides which KV blocks to keep for this layer — MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram
  2. To efficiently select the top- KV blocks, the index module ranks the index scores directly — MiniMax Sparse Attention
  3. Using I₁, I₂ from Step 1 as indices, we pull the corresponding block subsets out of the original K/V — MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram

Also called Top- block selection, top-k selection, TopK.

sources
2
model
1
lab adopt it
1
strongest
used

Documented in

MiniMax

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2
↑