Open Models Almanac
ModelsMiniMax-M3.1-Flash

MiniMax · usage rank #6

MiniMax-M3.1-Flash

What MiniMax-M3.1-Flash’s own documents say it is built from — every method with the sentence that describes it.

Parameters
~428B / ~23B active
Context
1M
Attention
sparse (indexer)MiniMax Sparse Attention (MSA) with GQA
Experts
128 routed + 1 shared, top-4
Input
text, image, video
Licence
MiniMax Community License
Released
2026-10-19

Rank by open-weight tokens on OpenRouter, week of 2026-10-26.

Against the consensus recipe

Its documents state 3 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 8 of the 19 architecture features checked — 7 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from Hugging Face Transformers (modeling_minimax_m3_vl.py, configuration_minimax_m3_vl.py) and config.json at revision ce7f4df87d, by anthropic/claude-sonnet-5 on 2026-11-07.

FeatureCodeIts documentsVerdict
Grouped-query attention code only in its codenot stated

64 query heads share 4 KV heads in every attention layer via repeat_kv grouping.

show evidencetext_config.num_attention_heads = 64text_config.num_key_value_heads = 4
self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
modeling_minimax_m3_vl.py · L406
self.q_proj = nn.Linear(config.hidden_size, config.num_attention_heads * self.head_dim, bias=False)
        self.k_proj = nn.Linear(config.hidden_size, config.num_key_value_heads * self.head_dim, bias=False)
        self.v_proj = nn.Linear(config.hidden_size, config.num_key_value_heads * self.head_dim, bias=False)
modeling_minimax_m3_vl.py · L410–412
Multi-head latent attention not in its codenot stated

Attention uses standard per-head q/k/v projections with GQA, with no latent kv_lora_rank/q_lora_rank down-projection mechanism in the code…

show the whole verdict

Attention uses standard per-head q/k/v projections with GQA, with no latent kv_lora_rank/q_lora_rank down-projection mechanism in the code or config.

Sliding-window attention not in its codenot stated

Layer types are only "full_attention" or the sparse indexer type; there is no sliding-window mechanism or config key enabling a local…

show evidence

Layer types are only "full_attention" or the sparse indexer type; there is no sliding-window mechanism or config key enabling a local window.

self.layer_types = [
                "minimax_m3_sparse" if f else "full_attention" for f in sparse_cfg["sparse_attention_freq"]
            ]
configuration_minimax_m3_vl.py · L142–144
Interleaved sliding-window and global attention not in its codenot stated

layer_types only distinguishes full_attention from the sparse-indexer attention type; there is no sliding-window layer variant interleaved…

show evidence

layer_types only distinguishes full_attention from the sparse-indexer attention type; there is no sliding-window layer variant interleaved with global attention.

self.layer_types = [
                "minimax_m3_sparse" if f else "full_attention" for f in sparse_cfg["sparse_attention_freq"]
            ]
configuration_minimax_m3_vl.py · L142–144
Indexer-selected sparse attention (DeepSeek Sparse Attention) code only in its codenot stated

A lightning indexer (MiniMaxM3VLIndexer) scores keys per block and each query attends only to its top index_topk_blocks blocks, enabled on…

show evidence

A lightning indexer (MiniMaxM3VLIndexer) scores keys per block and each query attends only to its top index_topk_blocks blocks, enabled on every layer via sparse_attention_freq all set to 1.

text_config.sparse_attention_config.use_sparse_attention = truetext_config.sparse_attention_config.sparse_topk_blocks = 16
topk = min(self.topk_blocks, num_key_blocks)
        topk_scores, topk_indices = block_scores.topk(topk, dim=-1)  # [B, H_idx, S_q, topk]
        return topk_indices.masked_fill(topk_scores == float("-inf"), -1)
modeling_minimax_m3_vl.py · L583–585
self.indexer = (
            MiniMaxM3VLIndexer(config, layer_idx) if config.layer_types[layer_idx] == "minimax_m3_sparse" else None
        )
modeling_minimax_m3_vl.py · L416–418
Linear-attention or state-space layers alongside full attention not in its codenot stated

No linear-attention, Mamba, or SSM layer type exists in this decoder; all layers are softmax attention (full or sparse-indexer selected).

Gated DeltaNet layers not in its codenot stated

There is no gated delta rule recurrent mixer implemented in the given files.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD state-space block exists in the given modeling code.

Learnable attention sink not in its codenot stated

The eager attention forward computes a plain softmax over query-key scores with no extra learned sink logit column.

show evidence
attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query.dtype)
modeling_minimax_m3_vl.py · L345
Gated attention output not in its codenot stated

attention_output_gate is explicitly false in text_config and no sigmoid gate multiplication on attention output appears in the attention…

show evidence

attention_output_gate is explicitly false in text_config and no sigmoid gate multiplication on attention output appears in the attention forward.

text_config.attention_output_gate = false
QK normalization code only in its codenot stated

Queries and keys are each normalized per head with RMSNorm before RoPE and the dot product, and use_qk_norm is true in this config.

show evidencetext_config.use_qk_norm = truetext_config.qk_norm_type = "per_head"
query_states = self.q_norm(self.q_proj(hidden_states).view(hidden_shape)).transpose(1, 2)
        key_states = self.k_norm(self.k_proj(hidden_states).view(hidden_shape)).transpose(1, 2)
modeling_minimax_m3_vl.py · L431–432
Partial RoPE code only in its codenot stated

RoPE is applied only to the first rotary_dim=64 of each 128-dim head (partial_rotary_factor 0.5), the rest passing through unrotated.

show evidencetext_config.rotary_dim = 64text_config.partial_rotary_factor = 0.5
rotary_dim = cos.shape[-1]
    q_rot, q_pass = q[..., :rotary_dim], q[..., rotary_dim:]
    k_rot, k_pass = k[..., :rotary_dim], k[..., rotary_dim:]
modeling_minimax_m3_vl.py · L375–377
YaRN RoPE scaling code unclearnot stated

rope_parameters is not populated with an explicit rope_scaling/type in the given config (only rope_theta and rotary_dim appear at top…

show the whole verdict

rope_parameters is not populated with an explicit rope_scaling/type in the given config (only rope_theta and rotary_dim appear at top level of text_config), so whether YaRN is switched on cannot be determined from what is given.

Layers without positional encoding (NoPE) not in its codenot stated

layer_types only toggles between full_attention and the sparse-indexer attention type; RoPE position_embeddings are passed to and applied…

show evidence

layer_types only toggles between full_attention and the sparse-indexer attention type; RoPE position_embeddings are passed to and applied in every decoder layer uniformly, with no layer lacking positional encoding.

cos, sin = position_embeddings
        query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
modeling_minimax_m3_vl.py · L435–436
Mixture of experts in its codeadopted

A sigmoid-scoring TopKRouter selects num_experts_per_tok=4 of num_local_experts=128 routed experts per token in the sparse MLP blocks.

show evidencetext_config.num_local_experts = 128text_config.num_experts_per_tok = 4
_, top_k_index = torch.topk(scores_for_choice, self.top_k, dim=-1, sorted=False)
        top_k_weights = routing_weights.gather(1, top_k_index)
modeling_minimax_m3_vl.py · L236–237
Shared expert code only in its codenot stated

Every MoE block also runs a dense shared expert (shared_intermediate_size=3072, n_shared_experts=1) whose output is added to the…

show evidence

Every MoE block also runs a dense shared expert (shared_intermediate_size=3072, n_shared_experts=1) whose output is added to the routed-expert output.

text_config.n_shared_experts = 1text_config.shared_intermediate_size = 3072
self.shared_experts = MiniMaxM3VLDenseMLP(config, intermediate_size=config.shared_intermediate_size)
modeling_minimax_m3_vl.py · L248
hidden_states = hidden_states * self.routed_scaling_factor
        hidden_states = hidden_states + shared_output
modeling_minimax_m3_vl.py · L258–259
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds a learned e_score_correction_bias to the sigmoid scores used only for top-k selection, not for the final weighting.

show evidencetext_config.use_routing_bias = true
scores_for_choice = routing_weights + self.e_score_correction_bias
        _, top_k_index = torch.topk(scores_for_choice, self.top_k, dim=-1, sorted=False)
        top_k_weights = routing_weights.gather(1, top_k_index)
modeling_minimax_m3_vl.py · L235–237
Multi-token prediction layers code unclearnot stated

The code references and skips loading MTP weights ("mtp.*") via a key-ignore pattern, but no mtp layer count config key (e.g.…

show evidence

The code references and skips loading MTP weights ("mtp.*") via a key-ignore pattern, but no mtp layer count config key (e.g. num_nextn_predict_layers) appears in the given config to confirm whether MTP modules are actually declared/built.

_keys_to_ignore_on_load_unexpected = [r"(^|\.)mtp\..*"]
modeling_minimax_m3_vl.py · L686
SwiGLU feed-forward code only in its codenot stated

Both the dense MLP and the MoE experts compute a SiLU-sigmoid-gated SwiGLU-style activation ((up+1)*gate*sigmoid(alpha*gate)) matching…

show evidence

Both the dense MLP and the MoE experts compute a SiLU-sigmoid-gated SwiGLU-style activation ((up+1)*gate*sigmoid(alpha*gate)) matching hidden_act swigluoai.

text_config.hidden_act = "swigluoai"
glu = gate * torch.sigmoid(gate * self.swiglu_alpha)
        return self.down_proj((up + 1.0) * glu)
modeling_minimax_m3_vl.py · L176–177

token mixer 3

sparse attention 3

Sparse attention core

“the first three full-attention layers are replaced with sparse attention”

Aggressive sparsity for inference efficiency used

“We have further optimized M3.1 Flash’s inference efficiency through more aggressive sparsity.”

block selection 1

MiniMax Sparse Attention core

“Attention: MiniMax Sparse Attention (MSA) throughout the model”

channel mixer 1

mixture of experts 1

Mixture of experts core

“M3.1-Flash retains M3’s core mixture-of-experts architecture”

quantization-aware training 1

Quantization-Aware Training used

“the routed MoE experts and the attention paths are quantized during training”

decoding strategy 3

DSpark used

“DSpark replaces the earlier EAGLE-like MTP approach.”

Temperature and nucleus sampling configuration optional

“Recommended sampling settings: temperature=1.0 and top_p=0.95.”

DSpark draft head optional

“The release includes the draft head for use with compatible serving implementations.”

reasoning control 1

Thinking mode optional

“For reasoning, set reasoning_effort in the chat template: low, medium, high, xhigh, or max.”

inference scheduling 1

Provider routing used

“we can recover by routing to another healthy provider”

1 more mentioned or evaluated
Load balancing across providers mentioned

inference scheduling “Learn more about our load balancing and provider routing options”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
minimax.io/blog/minimax-m31fofficial blogMiniMaxread
arxiv.org/abs/2606.13392technical reportMiniMaxread
huggingface.co/MiniMaxAI/MiniMax-M3.1-Flashmodel cardMiniMaxread
openrouter.ai/minimax/minimax-m3.1-flashvendor docsOpenRouterread
↑