Model techniques map
ModelsMiMo-V2.6-Pro

Xiaomi · released 2026-09-22

MiMo-V2.6-Pro

What MiMo-V2.6-Pro’s own documents say it is built from — every method with the sentence that describes it.

OpenRouterHugging Face

Curator’s noteFlagship of the MiMo-V2.6 series: sparse MoE, 1.02T total / 42B active, native omnimodal (text, image, video, audio), 1M context, hybrid SWA/global attention backbone with a 5-layer MTP drafter. The release is framed around RL: one mixed asynchronous GRPO run across domains and harnesses, groupwise agentic grading (GRS, GAR), and multi-prefix multi-teacher on-policy distillation (MOPD2). MIT. Shares the launch page and technical report with V2.6-Flash — same double-mention caveat as the V2.5 pair.

Against the consensus recipe

Its documents state 3 of the 15 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something the documents do not say, not something the model lacks.

  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 8 of the 19 architecture features checked — 7 of them are not stated in its documents.

Documents say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from the model repository’s own code (modeling_mimo_v2.py, configuration_mimo_v2.py) and config.json at revision 73875d00b3, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsEvidence
Grouped-query attention code only in its codenot stated

128 query heads share 8 key/value heads in full-attention layers (and 8 KV heads for the 128 SWA query heads too), implemented via repeat_kv grouping.

num_attention_heads = 128num_key_value_heads = 8
self.num_key_value_groups = self.num_attention_heads // self.num_key_value_heads
modeling_mimo_v2.py · L259
Multi-head latent attention not in its codenot stated

The attention module projects q/k/v directly per head with no low-rank latent down/up projection (no kv_lora_rank/q_lora_rank in config or code).

Sliding-window attention code only in its codenot stated

sliding_window is set to 128 and layers flagged 1 in hybrid_layer_pattern use is_swa attention with that window via create_sliding_window_causal_mask.

sliding_window = 128hybrid_layer_pattern = [0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1…
self.sliding_window = getattr(config, "sliding_window", None) if is_swa else None
modeling_mimo_v2.py · L262
Interleaved sliding-window and global attention code only in its codenot stated

hybrid_layer_pattern interleaves full-attention layers (0) with sliding-window layers (1) in a fixed repeating pattern across the 70 layers.

hybrid_layer_pattern = [0, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1…
is_swa_layer = config.hybrid_layer_pattern[layer_idx] == 1
        self.attention_type = "sliding_window_attention" if is_swa_layer else "full_attention"
modeling_mimo_v2.py · L400–401
Indexer-selected sparse attention (DeepSeek Sparse Attention) not in its codenot stated

No indexer scoring module or top-k token selection mechanism exists in the code, and no index_topk/index_n_heads config keys are present.

Linear-attention or state-space layers alongside full attention not in its codenot stated

All decoder layers use MiMoV2Attention (softmax attention, either full or sliding-window); there is no linear-attention or SSM mixer layer type.

Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent module is implemented; all token mixing is standard softmax attention.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD selective state-space block is implemented anywhere in the modeling file.

Learnable attention sink code only in its codenot stated

A learned per-head attention_sink_bias parameter is added as an extra logit column in the softmax denominator, enabled for both full and SWA layers via add_full_attention_sink_bias/add_swa_attention_sink_bias.

add_swa_attention_sink_bias = trueadd_full_attention_sink_bias = false
if sinks is not None:
        sinks = module.attention_sink_bias.reshape(1, -1, 1, 1).expand(query.shape[0], -1, query.shape[-2], -1)
        attn_weights = torch.cat([attn_weights, sinks], dim=-1)
modeling_mimo_v2.py · L89–91
Gated attention output not in its codenot stated

The attention output is passed directly to o_proj with no sigmoid gating multiplication before the output projection.

QK normalization not in its codenot stated

Query and key states are used directly after projection and RoPE with no RMSNorm/LayerNorm applied to them before the dot product.

Partial RoPE code only in its codenot stated

partial_rotary_factor of 0.334 makes rope_dim smaller than head_dim, so RoPE is applied only to a split portion of each query/key head with the rest (query_nope/key_nope) carrying no position signal.

partial_rotary_factor = 0.334rope_parameters.partial_rotary_factor = 0.334
query_rope, query_nope = query_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
        key_rope, key_nope = key_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
modeling_mimo_v2.py · L306–307
YaRN RoPE scaling not in its codenot stated

rope_parameters/rope_type is set to "default", not YaRN, so no YaRN scaling is applied.

rope_parameters.rope_type = "default"rope_parameters.type = "default"
Layers without positional encoding (NoPE) not in its codenot stated

All attention layers (full and SWA) apply the same partial RoPE scheme uniformly; there is no layer-specific toggle disabling RoPE entirely for some softmax attention layers.

Mixture of experts in its code core

MoE feed-forward layers route tokens to 8 of 384 experts per token via a learned sigmoid router (MiMoV2MoEGate), active per moe_layer_freq.

n_routed_experts = 384num_experts_per_tok = 8moe_layer_freq = [0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1…
self.experts = nn.ModuleList(
            [MiMoV2MLP(config, intermediate_size=config.moe_intermediate_size) for _ in range(config.n_routed_experts)]
        )
modeling_mimo_v2.py · L191–193
Shared expert not in its codenot stated

n_shared_experts is null in the config and no shared-expert module is instantiated in MiMoV2MoE.

n_shared_experts = null
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The MoE gate adds e_score_correction_bias to routing scores only for top-k selection (topk_method=noaux_tc) while the actual weighting uses the unbiased sigmoid scores.

topk_method = "noaux_tc"
scores_for_choice = scores.view(bsz * seq_len, -1) + self.e_score_correction_bias.unsqueeze(0)
modeling_mimo_v2.py · L164
topk_weight = scores.gather(1, topk_idx)
modeling_mimo_v2.py · L176
Multi-token prediction layers code unclear core

The code has a regex to ignore loading model.mtp.* weights implying an MTP module may exist, but no MTP module is defined in the given file and no mtp layer count key is present in config.json.

r"model\.mtp\..*",
modeling_mimo_v2.py · L1694
SwiGLU feed-forward code only in its codenot stated

Both dense and expert MLPs compute down_proj(act_fn(gate_proj(x)) * up_proj(x)) with hidden_act set to silu, giving SwiGLU.

hidden_act = "silu"
return self.down_proj(self.act_fn(self.gate_proj(hidden_states)) * self.up_proj(hidden_states))
modeling_mimo_v2.py · L132

model architecture 13

token mixer 4

channel mixer 3

mixture of experts 3

Mixture of Experts core
Sparse expert activation core

shared experts 1

No Shared Experts core

prediction head 1

Multi-Token Prediction core

multimodal architecture 5

Four-frame audio patches with within-patch bidirectional self-attention core
Two-stage audio encoding with tokenization and patch encoding core
Data-parallel-first multimodal encoding used
Joint full-model multimodal training used
Two-stage text-then-multimodal pre-training used

training objective 2

language modelling objective 1

Cross-entropy objective used

auxiliary loss 1

Loss masking used

optimization 8

filed at the root 1

Fused loss computation used

optimizer 1

Muown used

quantization-aware training 2

Quantize-Dequantize Training core
FP4 Quantization-Aware Training used

training stability 2

Behavioral regularization used
Safeguards against training drift and reward hacking used

training parallelism 1

Window-Bounded KV Exchange under Context Parallelism used

training runtime 1

CPU-resident optimizer states used

data curation 9

synthetic data 3

Large-scale environment synthesis and curation used
Multi-agent synthesis of local software mocks used
SFT teachers trained on synthetic demonstrations used

data mixture & curriculum 5

Sample Mixer core
Agent-centric multimodal data mixture used
Data Scheduler used
Dynamic sampling for mixed-task reinforcement learning used
Smooth weighted round-robin source scheduling used

sequence packing 1

Group-local packing used

post-training 77

supervised fine-tuning 3

SFT checkpoint for RL research core
Self-correction cold start used
SFT on MiMo-generated task data used

reinforcement learning algorithm 17

Penalty Module core
Dropping All-Zero-Advantage Groups default
Groupwise Advantage Redistribution used
Asynchronous Group Relative Policy Optimization used
Rollout Routing Replay used
Advantage Shaping used
Domain-Specific GRPO Training used
Dynamic Sampling used
Flagged-Token Masking and Penalty used
Freezing the MoE Router During Reinforcement Learning used
Group-Relative Length Penalty used
Hierarchical Penalty Escalation used
Keep Candidate-Set Replay used
Quality-Weighted Advantage Redistribution used
Single Mixed Reinforcement-Learning Run used
Single-Harness Reinforcement Learning used
Zero-Reward Correction for Leaked or External Answers used

reward modelling 14

Groupwise Agentic Grading core
Groupwise Reward Synthesis used
Adversarial screening used
Verifier cross-checking used
Behavior rubrics used
Five-dimension comparative grading of passing patches used
Hack-agent screening used
Monitoring-only penalty strategy used
Multiplicative reward synthesis used
Negative checks for unintended side effects used
Rule-based or model-judged trajectory error detection used
Segment-level behavioral penalties used
Solution rubrics used
Training-time trajectory auditing used

policy distillation 6

Multi-Prefix Multi-Teacher On-Policy Distillation used
Prefix-Conditioned On-Policy Distillation used
Autonomous Student Rollouts used
Distillation Fine-Tuning on MiMo-Generated Data used
Model Distillation used
Teacher-Trajectory and SFT-History Reuse used

rollout & RL infrastructure 28

Asynchronous reinforcement learning core
Agent Loop core
Agent-centric rollout execution core
Harness Pool core
Hierarchical trajectory data organization core
Payload Porter core
Predictive Rollout Dispatch core
Partial rollout used
Adaptive Rollout Concurrency used
Adaptive Rollout Scheduling used
Composite early-stop strategy used
Decoupled control plane and data plane used
Dialogue prefix matching used
Heterogeneous Agent Harnesses used
Incremental multimodal-delta transfer used
Isolated resettable rollout sandboxes used
Mixed-task rollout infrastructure used
Mixing tasks and harnesses in a single RL batch used
Multimodal rollout data handling used
Offloading blocking environment and tokenization work to background threads used
Per-source oversampling allocation used
Per-source priors for rollout sequence-length estimation used
Persistent host-actor pools used
Sample Mixer used
Sample replay used
Training–inference consistency mechanisms used
Asynchronous group-wise grading optional
Deficit-corrected scheduling evaluated

agentic post-training 7

Agentic reinforcement learning core
Environment hardening used
Container-level network isolation used
Environment preparation to prevent solution leakage used
Multi-agent task decomposition and coordination used
Multi-harness training used
Unified trajectory representation for agentic reinforcement learning used

mid-training & continual pretraining 2

Agent-centric mid-training used
Mid-training alignment data used

inference & serving 13

decoding strategy 3

DFlash core
Distribution-matched draft-model fine-tuning used
Throughput-based draft-block sizing used

reasoning control 1

Chain-of-thought reasoning used

KV cache management 3

Persistent per-dialogue-context KV caching core
Asynchronous cache offloading and restoration used
Hierarchical KV caching used

inference quantization 2

Dynamic activation scaling core
FP8 low-precision speculative draft computation used

serving parallelism 1

DeepEP used

inference scheduling 1

Greedy feasible-rank placement by remaining capacity used

agentic scaffolding 2

Mini-harnesses used
Request timeouts and retries for agent loops used

software implementation 3

inference engine 3

vLLM tensor parallelism used
vLLM optional
SGLang optional

evaluation 5

filed at the root 3

Behavior testing with stubs and edge cases used
Rollout-based auditing used
Visual checking of generated renders used

evaluation harness 1

Atomic binary rubric evaluation used

human & real-world evaluation 1

Same-prompt comparative DevOps testing used

other 3

filed at the root 3

Code-Driven Animation used
Generated Artifact Validation with Linters and Runtime Tests used
Training–Inference Consistency used

unfiled 10

Global attention with a dense feed-forward network in the first Transformer block core
MXFP4 weight storage with block size 32 core
Iterative render refinement toward a playable result used
Large collection of reinforcement-learning task environments used
Literature review, hypothesis formation, simulation, and candidate shortlisting for materials research used
Live reinforcement-learning training used
Pin API requests to the first-party provider and disable fallbacks used
Repeated test execution to detect flaky tests and reward instability used
Run ShellCheck at info severity to catch possible misspellings used
Use writable Kubernetes emptyDir volumes for nginx runtime directories used

Sources

The curated document list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

DocumentKindPublisherStatus
mimo.xiaomi.com/mimo-v2-6official blogXiaomi MiMonot fetched
huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/resolve/main/MiMo_V2_6_techn…technical reportXiaomi MiMoread
huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RLmodel cardXiaomi MiMoread
mimo.mi.com/models/en-US/mimo-v2.6-provendor docsXiaomi MiMoread
openrouter.ai/xiaomi/mimo-v2.6-provendor docsOpenRouterread
artificialanalysis.ai/models/mimo-v2-6-prothird party analysisArtificial Analysisread
computingforgeeks.com/xiaomi-mimo-v2-6-pro-flashthird party analysisComputingForGeeksread
venturebeat.com/technology/better-than-deepseek-xiaomis-mimo-v2-6-pro-d…newsVentureBeatnot fetched