Open Models Almanac
Methodsmodel architecturetoken mixersparse attentionsparse attention indexer

single mention · model architecture

Single-head index key

Single-head index key gives the sparse-attention index branch one shared key head that all heads use.1 Sharing the key makes the index-branch score computation nearly free while still indicating which KV blocks to examine.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. K_idx has only one head — all heads share the same index key — MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram
  2. As a result, Q_idx · K_idxᵀ costs almost nothing to compute — MiniMax Goes Sparse: Decoding M3's Attention from a Single Diagram

Also called single-head key for index branch, single-head K_idx.

Filed under model architecture › token mixer › sparse attention › sparse attention indexer