single mention · model architecture
Index Branch output
Index Branch output is a training mechanism that lets the sparse-attention Index Branch contribute an additional attention output to the layer.1 It attaches a value projection to the Index Branch and adds the result through a separate output projection, so the indexer learns block selection via next-token prediction.2
Trade-off It preserves short-context ability but gives little direct pressure on block selection, hurting long-context retrieval.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
The first mechanism lets the Index Branch contribute an additional attention output
— MiniMax Sparse Attentionwe attach a value projection to the Index Branch and compute . This output is added to the layer output through a separate output projection
— MiniMax Sparse AttentionLM Loss only preserves short-context ability but performs poorly on long-context retrieval
— MiniMax Sparse Attention
Also called Index Branch value head output.
Filed under model architecture › token mixer › sparse attention › sparse attention indexer