MiniMax-M3
MiniMaxWe introduce MiniMax Sparse Attention (MSA), a blockwise sparse attention built upon Grouped Query Attention (GQA).
6 more from this model
MSA is a clean and easily extensible new sparse attention architecture. It gives M3 a 1M context window
M3 introduces MiniMax Sparse Attention (MSA) to improve long context efficiency
The technical foundation is a new attention variant called MiniMax Sparse Attention (MSA).
MSA flips the logic and processes blocks sequentially.
Powered by the proprietary MiniMax Sparse Attention (MSA) architecture
M3 is powered by MiniMax Sparse Attention (MSA)