MiniMax-M3
MiniMaxadopting a "KV outer gather Q" approach that uses KV blocks as the outer loop to aggregate the queries that hit them
1 more from this model
organize sparse attention in a KV-outer order
implementation detail · filed under model architecture
A sparse-attention organization that loops over KV blocks to aggregate queries that select them.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See sparse attention for the methods filed beside it.
Also called KV outer gather Q.
2 quotes from 2 sources, one card per model, each opening with its strongest.
adopting a "KV outer gather Q" approach that uses KV blocks as the outer loop to aggregate the queries that hit them
organize sparse attention in a KV-outer order