implementation detail · filed under model architecture
Local Block
A fixed-pattern selection that always includes the local block; the evidence does not establish a more specific selection rule.
Also called forced sink and local window, forced sink and fixed local selection.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1not used 1
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
the local block containing is always selected as part of during both training and inference.
usedtraining objectivein MiniMax-M3MiniMax
the final recipe does not force the first block or a large local window
not usedmodel architecturein MiniMax Sparse AttentionMiniMax
Filed alongside
Other methods under model architecture :: token mixer :: sparse attention :: fixed-pattern sparse attention.