MiniMax · usage rank #6
MiniMax-M3.1-Flash
What MiniMax-M3.1-Flash’s own documents say it is built from — every method with the sentence that describes it.
- Parameters
- ~428B / ~23B active
- Context
- 1M
- Attention
- sparse (indexer)MiniMax Sparse Attention (MSA) with GQA
- Experts
- 128 routed + 1 shared, top-4
- Input
- text, image, video
- Licence
- MiniMax Community License
- Released
- 2026-10-19
Rank by open-weight tokens on OpenRouter, week of 2026-10-26.
Against the consensus recipe
Its documents state 3 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationnot in its documents
- Synthetic data generationnot in its documents
- Corpus deduplicationnot in its documents
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationnot in its documents
Model architecture
- Mixture of expertscore
- Multi-Token Predictionnot in its documents
- Grouped-query attentionnot in its documents
Optimization
- Muonnot in its documents
- Expert parallelismnot in its documents
- Quantization-Aware Trainingused
Post-training
- Supervised fine-tuningnot in its documents
- Asynchronous reinforcement learningnot in its documents
- Multi-Teacher On-Policy Distillationnot in its documents
Inference & serving
- Thinking modeoptional
- Multi-Token Predictionnot in its documents
- FP8not in its documents
Checked in its code
Its code shows 8 of the 19 architecture features checked — 7 of them are not stated in its documents.
Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.
Read from Hugging Face Transformers (modeling_minimax_m3_vl.py, configuration_minimax_m3_vl.py) and config.json at revision ce7f4df87d, by anthropic/claude-sonnet-5 on 2026-11-07.
| Feature | Code | Its documents | Verdict |
|---|---|---|---|
| Grouped-query attention code only | in its code | not stated | 64 query heads share 4 KV heads in every attention layer via repeat_kv grouping. show evidencetext_config.num_attention_heads = 64text_config.num_key_value_heads = 4 |
| Multi-head latent attention | not in its code | not stated | Attention uses standard per-head q/k/v projections with GQA, with no latent kv_lora_rank/q_lora_rank down-projection mechanism in the code… show the whole verdictAttention uses standard per-head q/k/v projections with GQA, with no latent kv_lora_rank/q_lora_rank down-projection mechanism in the code or config. |
| Sliding-window attention | not in its code | not stated | Layer types are only "full_attention" or the sparse indexer type; there is no sliding-window mechanism or config key enabling a local… show evidenceLayer types are only "full_attention" or the sparse indexer type; there is no sliding-window mechanism or config key enabling a local window. |
| Interleaved sliding-window and global attention | not in its code | not stated | layer_types only distinguishes full_attention from the sparse-indexer attention type; there is no sliding-window layer variant interleaved… show evidencelayer_types only distinguishes full_attention from the sparse-indexer attention type; there is no sliding-window layer variant interleaved with global attention. |
| Indexer-selected sparse attention (DeepSeek Sparse Attention) code only | in its code | not stated | A lightning indexer (MiniMaxM3VLIndexer) scores keys per block and each query attends only to its top index_topk_blocks blocks, enabled on… show evidenceA lightning indexer (MiniMaxM3VLIndexer) scores keys per block and each query attends only to its top index_topk_blocks blocks, enabled on every layer via sparse_attention_freq all set to 1. text_config.sparse_attention_config.use_sparse_attention = truetext_config.sparse_attention_config.sparse_topk_blocks = 16 |
| Linear-attention or state-space layers alongside full attention | not in its code | not stated | No linear-attention, Mamba, or SSM layer type exists in this decoder; all layers are softmax attention (full or sparse-indexer selected). |
| Gated DeltaNet layers | not in its code | not stated | There is no gated delta rule recurrent mixer implemented in the given files. |
| Mamba-2 layers | not in its code | not stated | No Mamba-2/SSD state-space block exists in the given modeling code. |
| Learnable attention sink | not in its code | not stated | The eager attention forward computes a plain softmax over query-key scores with no extra learned sink logit column. show evidence |
| Gated attention output | not in its code | not stated | attention_output_gate is explicitly false in text_config and no sigmoid gate multiplication on attention output appears in the attention… show evidenceattention_output_gate is explicitly false in text_config and no sigmoid gate multiplication on attention output appears in the attention forward. text_config.attention_output_gate = false |
| QK normalization code only | in its code | not stated | Queries and keys are each normalized per head with RMSNorm before RoPE and the dot product, and use_qk_norm is true in this config. show evidencetext_config.use_qk_norm = truetext_config.qk_norm_type = "per_head" |
| Partial RoPE code only | in its code | not stated | RoPE is applied only to the first rotary_dim=64 of each 128-dim head (partial_rotary_factor 0.5), the rest passing through unrotated. show evidencetext_config.rotary_dim = 64text_config.partial_rotary_factor = 0.5 |
| YaRN RoPE scaling | code unclear | not stated | rope_parameters is not populated with an explicit rope_scaling/type in the given config (only rope_theta and rotary_dim appear at top… show the whole verdictrope_parameters is not populated with an explicit rope_scaling/type in the given config (only rope_theta and rotary_dim appear at top level of text_config), so whether YaRN is switched on cannot be determined from what is given. |
| Layers without positional encoding (NoPE) | not in its code | not stated | layer_types only toggles between full_attention and the sparse-indexer attention type; RoPE position_embeddings are passed to and applied… show evidencelayer_types only toggles between full_attention and the sparse-indexer attention type; RoPE position_embeddings are passed to and applied in every decoder layer uniformly, with no layer lacking positional encoding. |
| Mixture of experts | in its code | adopted | A sigmoid-scoring TopKRouter selects num_experts_per_tok=4 of num_local_experts=128 routed experts per token in the sparse MLP blocks. show evidencetext_config.num_local_experts = 128text_config.num_experts_per_tok = 4 |
| Shared expert code only | in its code | not stated | Every MoE block also runs a dense shared expert (shared_intermediate_size=3072, n_shared_experts=1) whose output is added to the… show evidenceEvery MoE block also runs a dense shared expert (shared_intermediate_size=3072, n_shared_experts=1) whose output is added to the routed-expert output. text_config.n_shared_experts = 1text_config.shared_intermediate_size = 3072 |
| Auxiliary-loss-free load balancing (selection bias) code only | in its code | not stated | The router adds a learned e_score_correction_bias to the sigmoid scores used only for top-k selection, not for the final weighting. show evidencetext_config.use_routing_bias = true |
| Multi-token prediction layers | code unclear | not stated | The code references and skips loading MTP weights ("mtp.*") via a key-ignore pattern, but no mtp layer count config key (e.g.… show evidenceThe code references and skips loading MTP weights ("mtp.*") via a key-ignore pattern, but no mtp layer count config key (e.g. num_nextn_predict_layers) appears in the given config to confirm whether MTP modules are actually declared/built. |
| SwiGLU feed-forward code only | in its code | not stated | Both the dense MLP and the MoE experts compute a SiLU-sigmoid-gated SwiGLU-style activation ((up+1)*gate*sigmoid(alpha*gate)) matching… show evidenceBoth the dense MLP and the MoE experts compute a SiLU-sigmoid-gated SwiGLU-style activation ((up+1)*gate*sigmoid(alpha*gate)) matching hidden_act swigluoai. text_config.hidden_act = "swigluoai" |
token mixer 3
sparse attention 3
“the first three full-attention layers are replaced with sparse attention”
“We have further optimized M3.1 Flash’s inference efficiency through more aggressive sparsity.”
block selection 1
“Attention: MiniMax Sparse Attention (MSA) throughout the model”
channel mixer 1
mixture of experts 1
“M3.1-Flash retains M3’s core mixture-of-experts architecture”
quantization-aware training 1
“the routed MoE experts and the attention paths are quantized during training”
decoding strategy 3
“DSpark replaces the earlier EAGLE-like MTP approach.”
“Recommended sampling settings: temperature=1.0 and top_p=0.95.”
“The release includes the draft head for use with compatible serving implementations.”
reasoning control 1
“For reasoning, set reasoning_effort in the chat template: low, medium, high, xhigh, or max.”
inference scheduling 1
“we can recover by routing to another healthy provider”
1 more mentioned or evaluated
inference scheduling “Learn more about our load balancing and provider routing options”
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| minimax.io/blog/minimax-m31f | official blog | MiniMax | read |
| arxiv.org/abs/2606.13392 | technical report | MiniMax | read |
| huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash | model card | MiniMax | read |
| openrouter.ai/minimax/minimax-m3.1-flash | vendor docs | OpenRouter | read |