Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionsparse attention indexer

single source · model architecture

Index Branch

The Index Branch is a lightweight component that selects a small set of key blocks from the causal context for each query token.1 It operates at block granularity by first scoring visible key tokens and then aggregating these scores to the block level.2

Trade-off The method reduces the per-query attention cost, which is fixed as the sequence length increases.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. For each query token, a lightweight Index Branch selects a small set of key blocks from the causal context — MiniMax Sparse Attention
  2. the Index Branch first scores visible key tokens, then aggregates these scores to the block level — MiniMax Sparse Attention
  3. the per-query attention cost is reduced from to , which is fixed as the sequence length increases — MiniMax Sparse Attention

Filed under model architecture › token mixer › sparse attention › sparse attention indexer