Model techniques map
ModelsMiMo-V2.6-Flash

Xiaomi · usage rank #6 · released 2026-09-22

MiMo-V2.6-Flash

What MiMo-V2.6-Flash’s own documents say it is built from — every method with the sentence that describes it.

OpenRouterHugging Face

Curator’s noteThe cost-efficient sibling of MiMo-V2.6-Pro: sparse MoE, 309B total / 15B active (48 layers, 39 SWA / 9 global; 256 routed experts, 8 active), the same omnimodal encoders, 1M context and RL recipe. MIT. Launch page and technical report are shared with V2.6-Pro. The MiMo-V2.6-Distill-Qwen-9B released alongside is not on OpenRouter and is not covered.

Rank by open-weight tokens on OpenRouter, week of 2026-09-21.

Against the consensus recipe

Its documents state 4 of the 15 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something the documents do not say, not something the model lacks.

  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 9 of the 19 architecture features checked — 7 of them are not stated in its documents.

Documents say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from the model repository’s own code (modeling_mimo_v2.py, configuration_mimo_v2.py) and config.json at revision 5711b26816, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsEvidence
Grouped-query attention code only in its codenot stated

Full-attention layers use 64 query heads with 4 KV heads (and SWA layers use 64 query heads with 8 KV heads), repeated via repeat_kv, giving grouped-query attention throughout.

num_attention_heads = 64num_key_value_heads = 4swa_num_key_value_heads = 8
self.num_key_value_groups = self.num_attention_heads // self.num_key_value_heads
modeling_mimo_v2.py · L259
key_states = repeat_kv(key, module.num_key_value_groups)
    value_states = repeat_kv(value, module.num_key_value_groups)
modeling_mimo_v2.py · L82–83
Multi-head latent attention not in its codenot stated

There is no low-rank KV latent projection (no kv_lora_rank/q_lora_rank in config or code); attention uses standard per-head q/k/v projections with GQA instead.

Sliding-window attention code only in its codenot stated

Layers flagged 1 in hybrid_layer_pattern are is_swa layers whose sliding_window is set to the non-null config value 128, and a sliding-window causal mask is built for them.

sliding_window = 128hybrid_layer_pattern = [0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1…
self.sliding_window = getattr(config, "sliding_window", None) if is_swa else None
modeling_mimo_v2.py · L262
causal_mask_mapping["sliding_window_attention"] = create_sliding_window_causal_mask(**mask_kwargs)
modeling_mimo_v2.py · L1657
Interleaved sliding-window and global attention code only in its codenot stated

hybrid_layer_pattern interleaves full-attention layers (0) with sliding-window layers (1) in a fixed repeating pattern across the 48 decoder layers.

hybrid_layer_pattern = [0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1…
is_swa_layer = config.hybrid_layer_pattern[layer_idx] == 1
        self.attention_type = "sliding_window_attention" if is_swa_layer else "full_attention"
modeling_mimo_v2.py · L400–401
Indexer-selected sparse attention (DeepSeek Sparse Attention) not in its codenot stated

No indexer scoring or top-k token selection mechanism exists in the code, and no index_topk/index_n_heads config keys are present.

Linear-attention or state-space layers alongside full attention not in its codenot stated

All token-mixing layers in this file are softmax attention (full or sliding-window); there is no linear-attention, Mamba, or state-space mixer implementation.

Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent mixer is implemented; only softmax attention layers exist.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD selective state-space block is implemented anywhere in the given files.

Learnable attention sink code only in its codenot stated

Language-model attention has a learned per-head attention_sink_bias parameter appended as an extra logit column in eager attention softmax, enabled for SWA layers by add_swa_attention_sink_bias=true.

add_swa_attention_sink_bias = trueadd_full_attention_sink_bias = false
if sinks is not None:
        sinks = module.attention_sink_bias.reshape(1, -1, 1, 1).expand(query.shape[0], -1, query.shape[-2], -1)
        attn_weights = torch.cat([attn_weights, sinks], dim=-1)
modeling_mimo_v2.py · L89–91
self.attention_sink_bias = (
            nn.Parameter(torch.empty(self.num_attention_heads), requires_grad=False)
            if (
                (getattr(config, "add_full_attention_sink_bias", False) and not is_swa)
                or (getattr(config, "add_swa_attention_sink_bias", False) and is_swa)
            )
            else None
        )
modeling_mimo_v2.py · L268–275
Gated attention output not in its codenot stated

The language-model attention output goes straight through o_proj with no sigmoid gate multiplying it; no attn_output_gate/use_output_gate config or code exists for the LM.

QK normalization not in its codenot stated

Query and key tensors are used directly for RoPE and attention without any RMSNorm/LayerNorm normalization step in MiMoV2Attention.

Partial RoPE code only in its codenot stated

Each head's dimension is split into a rotary part (rope_dim, sized by partial_rotary_factor 0.334) and a non-rotary part (head_dim - rope_dim) that carries no position signal.

partial_rotary_factor = 0.334head_dim = 192
self.rope_dim = int(self.head_dim * getattr(config, "partial_rotary_factor", 1.0))
modeling_mimo_v2.py · L253
query_rope, query_nope = query_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
        key_rope, key_nope = key_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
modeling_mimo_v2.py · L306–307
YaRN RoPE scaling not in its codenot stated

rope_parameters/rope_type is set to "default" in this config, not YaRN, so no YaRN scaling is applied.

rope_parameters.rope_type = "default"rope_parameters.type = "default"
Layers without positional encoding (NoPE) not in its codenot stated

Every attention layer (full or sliding-window) applies rotary embeddings via its own rotary embedding module; there is no layer that skips RoPE entirely for softmax attention.

Mixture of experts in its code core

MoE feed-forward layers route each token via a sigmoid-scored top-8-of-256 router (MiMoV2MoEGate/MiMoV2MoE) applied on layers flagged in moe_layer_freq.

n_routed_experts = 256num_experts_per_tok = 8
self.experts = nn.ModuleList(
            [MiMoV2MLP(config, intermediate_size=config.moe_intermediate_size) for _ in range(config.n_routed_experts)]
        )
modeling_mimo_v2.py · L191–193
_, topk_idx = torch.topk(tmp_scores, k=self.top_k, dim=-1, sorted=False)
            topk_weight = scores.gather(1, topk_idx)
modeling_mimo_v2.py · L175–176
Shared expert not in its codenot stated

n_shared_experts is null in this config, and MiMoV2MoE builds no additional always-on expert module.

n_shared_experts = null
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds e_score_correction_bias to scores only for top-k selection (noaux_tc topk_method), while topk_weight for weighting is gathered from the unbiased sigmoid scores.

topk_method = "noaux_tc"
scores_for_choice = scores.view(bsz * seq_len, -1) + self.e_score_correction_bias.unsqueeze(0)
modeling_mimo_v2.py · L164
_, topk_idx = torch.topk(tmp_scores, k=self.top_k, dim=-1, sorted=False)
            topk_weight = scores.gather(1, topk_idx)
modeling_mimo_v2.py · L175–176
Multi-token prediction layers in its code core

config declares num_nextn_predict_layers=3 for MTP modules, even though the given modeling file does not build them (weights are ignored via the mtp key pattern), which counts per the check's rule for config-declared nonzero MTP counts.

num_nextn_predict_layers = 3
r"model\.mtp\..*",
modeling_mimo_v2.py · L1694
SwiGLU feed-forward code only in its codenot stated

MiMoV2MLP (used both as dense MLP and MoE expert) computes down_proj(act_fn(gate_proj(x)) * up_proj(x)) with hidden_act=silu, i.e. SwiGLU.

hidden_act = "silu"
return self.down_proj(self.act_fn(self.gate_proj(hidden_states)) * self.up_proj(hidden_states))
modeling_mimo_v2.py · L132

model architecture 14

token mixer 4

channel mixer 2

mixture of experts 2

Mixture of Experts core

shared experts 1

No Shared Experts core

prediction head 1

Multi-Token Prediction core

multimodal architecture 6

Four-frame audio patches with within-patch bidirectional self-attention core
Joint multimodal input and understanding core
Two-stage audio encoding with tokenization and patch encoding core
Data-parallel-first multimodal encoding used
Joint full-model multimodal training used
Two-stage text-then-multimodal pre-training used

context capacity 1

1M-token context window core

training objective 2

language modelling objective 1

Cross-entropy objective used

auxiliary loss 1

Loss masking used

optimization 8

filed at the root 1

Fused loss computation used

optimizer 1

Muown used

quantization-aware training 2

Quantize-Dequantize Training core
FP4 Quantization-Aware Training used

training stability 2

Behavioral regularization used
Safeguards against training drift and reward hacking used

training parallelism 1

Window-Bounded KV Exchange under Context Parallelism used

training runtime 1

CPU-resident optimizer states used

data curation 9

synthetic data 3

Large-scale environment synthesis and curation used
Multi-agent synthesis of local software mocks used
SFT teachers trained on synthetic demonstrations used

data mixture & curriculum 5

Sample Mixer core
Agent-centric multimodal data mixture used
Data Scheduler used
Dynamic sampling for mixed-task reinforcement learning used
Smooth weighted round-robin source scheduling used

sequence packing 1

Group-local packing used

post-training 73

supervised fine-tuning 3

SFT checkpoint for RL research core
Self-correction cold start used
SFT on MiMo-generated task data used

reinforcement learning algorithm 16

Groupwise Advantage Redistribution core
Asynchronous Group Relative Policy Optimization core
Penalty Module core
Dropping All-Zero-Advantage Groups default
Rollout Routing Replay used
Advantage Shaping used
Domain-Specific GRPO Training used
Dynamic Sampling used
Flagged-Token Masking and Penalty used
Freezing the MoE Router During Reinforcement Learning used
Group-Relative Length Penalty used
Hierarchical Penalty Escalation used
Keep Candidate-Set Replay used
Quality-Weighted Advantage Redistribution used
Single-Harness Reinforcement Learning used
Zero-Reward Correction for Leaked or External Answers used

reward modelling 14

Groupwise Agentic Grading core
Groupwise Reward Synthesis core
Adversarial screening used
Verifier cross-checking used
Behavior rubrics used
Five-dimension comparative grading of passing patches used
Hack-agent screening used
Monitoring-only penalty strategy used
Multiplicative reward synthesis used
Negative checks for unintended side effects used
Rule-based or model-judged trajectory error detection used
Segment-level behavioral penalties used
Solution rubrics used
Training-time trajectory auditing used

policy distillation 5

Multi-Prefix Multi-Teacher On-Policy Distillation used
Prefix-Conditioned On-Policy Distillation used
Autonomous Student Rollouts used
Distillation Fine-Tuning on MiMo-Generated Data used
Model Distillation used

rollout & RL infrastructure 27

Asynchronous reinforcement learning core
Agent Loop core
Agent-centric rollout execution core
Harness Pool core
Hierarchical trajectory data organization core
Payload Porter core
Predictive Rollout Dispatch core
Partial rollout used
Adaptive Rollout Concurrency used
Adaptive Rollout Scheduling used
Composite early-stop strategy used
Decoupled control plane and data plane used
Dialogue prefix matching used
Heterogeneous Agent Harnesses used
Incremental multimodal-delta transfer used
Isolated resettable rollout sandboxes used
Mixed-task rollout infrastructure used
Multimodal rollout data handling used
Offloading blocking environment and tokenization work to background threads used
Per-source oversampling allocation used
Per-source priors for rollout sequence-length estimation used
Persistent host-actor pools used
Sample Mixer used
Sample replay used
Training–inference consistency mechanisms used
Asynchronous group-wise grading optional
Deficit-corrected scheduling evaluated

agentic post-training 6

Agentic reinforcement learning core
Environment hardening used
Container-level network isolation used
Environment preparation to prevent solution leakage used
Multi-harness training used
Unified trajectory representation for agentic reinforcement learning used

mid-training & continual pretraining 2

Agent-centric mid-training used
Mid-training alignment data used

inference & serving 15

decoding strategy 5

DFlash core
Distribution-matched draft-model fine-tuning used
Throughput-based draft-block sizing used
Speculative decoding optional
EAGLE optional

KV cache management 3

Persistent per-dialogue-context KV caching core
Asynchronous cache offloading and restoration used
Hierarchical KV caching used

inference quantization 2

Dynamic activation scaling core
FP8 low-precision speculative draft computation used

serving parallelism 1

DeepEP used

inference scheduling 1

Greedy feasible-rank placement by remaining capacity used

agentic scaffolding 3

Mini-harnesses used
Request timeouts and retries for agent loops used
Multi-agent collaboration mentioned

software implementation 1

inference engine 1

vLLM tensor parallelism used

evaluation 4

filed at the root 2

Behavior testing with stubs and edge cases used
Rollout-based auditing used

evaluation harness 1

Atomic binary rubric evaluation used

human & real-world evaluation 1

Same-prompt comparative DevOps testing used

other 2

filed at the root 2

Generated Artifact Validation with Linters and Runtime Tests used
Training–Inference Consistency used

unfiled 8

MXFP4 weight storage with block size 32 core
Large collection of reinforcement-learning task environments used
Live reinforcement-learning training used
Optimization on real agent-harness environments used
Pin API requests to the first-party provider and disable fallbacks used
Repeated test execution to detect flaky tests and reward instability used
Run ShellCheck at info severity to catch possible misspellings used
Use writable Kubernetes emptyDir volumes for nginx runtime directories used

Sources

The curated document list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

DocumentKindPublisherStatus
mimo.xiaomi.com/mimo-v2-6official blogXiaomi MiMonot fetched
huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/resolve/main/MiMo_V2_6_techn…technical reportXiaomi MiMoread
huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RLmodel cardXiaomi MiMoread
mimo.mi.com/models/en-US/mimo-v2.6-flashvendor docsXiaomi MiMoread
openrouter.ai/xiaomi/mimo-v2.6-flashvendor docsOpenRouterread
computingforgeeks.com/xiaomi-mimo-v2-6-pro-flashthird party analysisComputingForGeeksread