implementation detail · filed under model architecture
Top-k block selection
Selects the top-k key/value blocks, with max-pooling scoring specified in one mention.
Also called Top- block selection, top-k selection, TopK.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 2
Documented in
Evidence
2 spans quoted from the sources, strongest treatment first.
selects the top- blocks via max-pooling scoring
usedmodel architecturein MiniMax-M3MiniMax
TopK decides which KV blocks to keep for this layer and this GQA group
usedmodel architecturein MiniMax-M3MiniMax
Filed alongside
Other methods under model architecture :: token mixer :: sparse attention :: block selection.