specific method · filed under model architecture
MiniMax Sparse Attention
A blockwise sparse-attention architecture built on Grouped Query Attention for efficient long-context processing.
Also called MSA (MiniMax Sparse Attention), MiniMax Sparse Attention (MSA).
- sources
- 6
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- MiniMax Sparse Attention (MiniMax AI) paper arxiv.org
Evidence
6 spans quoted from the sources, strongest treatment first.
MSA is a clean and easily extensible new sparse attention architecture. It gives M3 a 1M context window
We introduce MiniMax Sparse Attention (MSA), a blockwise sparse attention built upon Grouped Query Attention (GQA).
M3 introduces MiniMax Sparse Attention (MSA) to improve long context efficiency
The technical foundation is a new attention variant called MiniMax Sparse Attention (MSA).
M3 is powered by MiniMax Sparse Attention (MSA)
Powered by the proprietary MiniMax Sparse Attention (MSA) architecture
Filed alongside
Other methods under model architecture :: token mixer :: sparse attention :: block selection.