Model techniques map
ModelsMiMo-V2.5-Pro

Xiaomi · released 2026-04-22

MiMo-V2.5-Pro

What MiMo-V2.5-Pro’s own documents say it is built from — every method with the sentence that describes it.

OpenRouterHugging Face

Curator’s noteLarger sibling launched alongside MiMo-V2.5; MIT license, 1M context. Shares the mimo.xiaomi.com launch page with V2.5 — same double-mention caveat as the DeepSeek V4 pair.

Against the consensus recipe

Its documents state 7 of the 15 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something the documents do not say, not something the model lacks.

  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 8 of the 19 architecture features checked — 4 of them are not stated in its documents.

Documents say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from the model repository’s own code (modeling_mimo_v2.py, configuration_mimo_v2.py) and config.json at revision 21d1ecfecd, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsEvidence
Grouped-query attention code only in its codenot stated

Full attention layers use 128 query heads sharing 8 key/value heads via repeat_kv grouping.

num_attention_heads = 128num_key_value_heads = 8
self.num_key_value_groups = self.num_attention_heads // self.num_key_value_heads
modeling_mimo_v2.py · L254
Multi-head latent attention not in its codenot stated

No latent KV down/up projection mechanism (kv_lora_rank/q_lora_rank) exists in the code or config; attention uses standard per-head q/k/v projections.

Sliding-window attention in its code used

SWA layers (marked by hybrid_layer_pattern==1) use sliding_window=128 to build a sliding-window causal mask.

sliding_window = 128
self.sliding_window = getattr(config, "sliding_window", None) if is_swa else None
modeling_mimo_v2.py · L257
Interleaved sliding-window and global attention in its code used

hybrid_layer_pattern interleaves full-attention layers (0) with sliding-window layers (1) in a fixed repeating pattern across 70 layers.

is_swa_layer = config.hybrid_layer_pattern[layer_idx] == 1
        self.attention_type = "sliding_window_attention" if is_swa_layer else "full_attention"
modeling_mimo_v2.py · L395–396
Indexer-selected sparse attention (DeepSeek Sparse Attention) not in its codenot stated

No indexer-scoring or top-k token selection mechanism appears in the code or config (no index_topk/index_n_heads keys).

Linear-attention or state-space layers alongside full attention not in its codenot stated

All decoder layers use MiMoV2Attention (softmax attention, either full or sliding-window); there is no linear-attention or SSM mixer layer type.

self.self_attn = MiMoV2Attention(
            config, is_swa_layer, layer_idx, projection_layout=attention_projection_layout
        )
modeling_mimo_v2.py · L397–399
Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent mixer implementation or config keys (linear_num_key_heads, linear_conv_kernel_dim) are present.

Mamba-2 layers not in its codenot stated

No Mamba-2 SSD state-space block implementation or related config keys (mamba_num_heads, ssm_state_size) are present.

Learnable attention sink in its code used

Sliding-window layers add a learned per-head attention_sink_bias as an extra logit column in the softmax denominator (enabled via add_swa_attention_sink_bias=true).

add_swa_attention_sink_bias = trueadd_full_attention_sink_bias = false
if sinks is not None:
        sinks = module.attention_sink_bias.reshape(1, -1, 1, 1).expand(query.shape[0], -1, query.shape[-2], -1)
        attn_weights = torch.cat([attn_weights, sinks], dim=-1)
modeling_mimo_v2.py · L84–86
self.attention_sink_bias = (
            nn.Parameter(torch.empty(self.num_attention_heads), requires_grad=False)
            if (
                (getattr(config, "add_full_attention_sink_bias", False) and not is_swa)
                or (getattr(config, "add_swa_attention_sink_bias", False) and is_swa)
            )
modeling_mimo_v2.py · L263–268
Gated attention output not in its codenot stated

No sigmoid gating multiplication is applied to the attention output before o_proj; attn_output goes directly into o_proj.

attn_output = attn_output.reshape(*input_shape, -1).contiguous()
        attn_output = self.o_proj(attn_output)
modeling_mimo_v2.py · L349–350
QK normalization not in its codenot stated

No RMSNorm/LayerNorm is applied to query or key states before the attention dot product in MiMoV2Attention; queries/keys go straight from projection/RoPE into matmul.

Partial RoPE code only in its codenot stated

Only rope_dim = head_dim * partial_rotary_factor (0.334) of each head is rotated, the remainder (query_nope/key_nope) carries no position signal.

partial_rotary_factor = 0.334
query_rope, query_nope = query_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
        key_rope, key_nope = key_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
modeling_mimo_v2.py · L301–302
YaRN RoPE scaling not in its codenot stated

rope_scaling is not set in this config (no key present) so no YaRN scaling is applied; rotary embedding falls back to the default type.

self.rope_type = "default"
modeling_mimo_v2.py · L448
Layers without positional encoding (NoPE) not in its codenot stated

All softmax-attention layers apply the same partial_rotary_factor RoPE scheme uniformly (full-attention and sliding-window layers both get rotary embeddings via rotary_emb/swa_rotary_emb); there is no layer subset entirely without positional encoding.

position_embeddings = self.rotary_emb(hidden_states, position_ids)
        swa_position_embeddings = self.swa_rotary_emb(hidden_states, position_ids)
modeling_mimo_v2.py · L589–590
Mixture of experts in its code core

Feed-forward layers route tokens to 8-of-384 experts chosen by a learned sigmoid router (noaux_tc top-k selection).

n_routed_experts = 384num_experts_per_tok = 8
self.experts = nn.ModuleList(
            [MiMoV2MLP(config, intermediate_size=config.moe_intermediate_size) for _ in range(config.n_routed_experts)]
        )
modeling_mimo_v2.py · L186–188
Shared expert not in its codenot stated

n_shared_experts is null in this config and no shared-expert module is instantiated in MiMoV2MoE.

n_shared_experts = null
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The MoE gate adds e_score_correction_bias to routing scores only for top-k selection (topk_method noaux_tc), not for weighting.

topk_method = "noaux_tc"
scores_for_choice = scores.view(bsz * seq_len, -1) + self.e_score_correction_bias.unsqueeze(0)
modeling_mimo_v2.py · L159
Multi-token prediction layers code unclear used

The code ignores model.mtp.* weights on load and the config has no nextn/mtp layer count key, so it cannot be determined whether MTP modules are declared for this checkpoint.

r"model\.mtp\..*",
modeling_mimo_v2.py · L623
SwiGLU feed-forward code only in its codenot stated

MLP and expert feed-forwards compute silu(gate_proj(x)) * up_proj(x) then down_proj, with hidden_act set to silu.

hidden_act = "silu"
return self.down_proj(self.act_fn(self.gate_proj(hidden_states)) * self.up_proj(hidden_states))
modeling_mimo_v2.py · L127

model architecture 13

token mixer 5

softmax attention 4

global attention 1

Global attention used

attention sink 1

Learnable attention sink bias used

hybrid layer stacking 1

Hybrid Attention core

channel mixer 1

mixture of experts 1

Mixture of Experts core

prediction head 1

Multi-Token Prediction used

multimodal architecture 4

Native multimodal understanding core
Native visual and audio understanding core
Projector warmup used
Visual and audio encoders with lightweight projectors used

context capacity 2

1M-token context window used
Progressive context extension used

training objective 2

language modelling objective 1

Text pre-training used

multi-token prediction objective 1

Multi-Token Prediction used

optimization 1

training precision 1

FP8 mixed-precision training used

data curation 2

data filtering 1

Multi-dimensional data filtering used

data mixture & curriculum 1

Three-stage data mixture strategy used

post-training 11

supervised fine-tuning 2

Supervised fine-tuning used
Cold-started SFT model used

reinforcement learning algorithm 1

Reinforcement Learning used

reward modelling 2

Rule-based verifier used
Test-difficulty-driven code reward used

policy distillation 1

Multi-Teacher On-Policy Distillation core

rollout & RL infrastructure 3

Seamless Rollout Engine used
Continuous rollout with asynchronous reward computation and early termination used
Data re-sampling strategy used

agentic post-training 2

Agentic reinforcement learning used
Agentic post-training used

inference & serving 5

decoding strategy 4

Speculative decoding used
Multi-Token Prediction used
EAGLE optional
Multi-layer EAGLE optional

serving parallelism 1

DeepEP optional

software implementation 3

agent product 3

OpenCode optional
Claude Code optional
Kilo optional

unfiled 4

agentic model core
harness awareness core
multimodal pre-training used
ngspice simulation used

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

Sources

The curated document list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

DocumentKindPublisherStatus
mimo.xiaomi.com/mimo-v2-5official blogXiaomi MiMoread
huggingface.co/XiaomiMiMo/MiMo-V2.5-Promodel cardXiaomi MiMoread
github.com/XiaomiMiMo/MiMocode repoXiaomi MiMoread
openrouter.ai/xiaomi/mimo-v2.5-provendor docsOpenRouterread
artificialanalysis.ai/models/mimo-v2-5-prothird party analysisArtificial Analysisread
marktechpost.com/2026/04/22/xiaomi-releases-mimo-v2-5-pro-and-mimo-v2-5…newsMarkTechPostread
buildfastwithai.com/blogs/xiaomi-mimo-v2-5-pro-review-2026third party analysisBuildFastWithAIread