implementation detail · filed under model architecture
Index Branch output
The output of the Index Branch, which is added to the layer output.
Also called Index Branch value head output.
- source
- 1
- model
- 1
- labs adopt it
- 0
- strongest
- not used
How sources treat it
One count per evidence span, weakest treatment to strongest.
not used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
the Index Branch output is added to the layer output
not usedmodel architecturein MiniMax Sparse AttentionMiniMax
Filed alongside
Other methods under model architecture :: token mixer :: sparse attention :: sparse attention indexer.
Compressed Sparse Attention 2Hierarchical Sparse IndexerIndexCacheIndexShareLightning IndexerReindex ModeReuse ModeDense Warm-up StageFine-grained token selectionIndex BranchIndexer WarmupReuse QSA index selection across speculative decoding stepsAverage poolingBlock-causal scoringCompressed lightweight indexerCross-stage shared-state management for attention reuseDetached indexer-input optimizationIndex Branch value headIndexPoolMQA indexerSingle-head index key