Open Models Almanac
ModelsMiMo-V2.5-Pro

Xiaomi

MiMo-V2.5-Pro

What MiMo-V2.5-Pro’s own documents say it is built from — every method with the sentence that describes it.

Parameters
1.02T / 42B active
Context
1M
Attention
sliding/global hybridhybrid sliding-window (128) + global GQA, learnable attention sink
Experts
384 routed, top-8, no shared
Input
text
Licence
MIT
Released
2026-04-22

Against the consensus recipe

Its documents state 6 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 8 of the 19 architecture features checked — 4 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from the model repository’s own code (modeling_mimo_v2.py, configuration_mimo_v2.py) and config.json at revision 21d1ecfecd, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsVerdict
Grouped-query attention code only in its codenot stated

Full attention layers use 128 query heads sharing 8 key/value heads via repeat_kv grouping.

show evidencenum_attention_heads = 128num_key_value_heads = 8
self.num_key_value_groups = self.num_attention_heads // self.num_key_value_heads
modeling_mimo_v2.py · L254
Multi-head latent attention not in its codenot stated

No latent KV down/up projection mechanism (kv_lora_rank/q_lora_rank) exists in the code or config; attention uses standard per-head q/k/v…

show the whole verdict

No latent KV down/up projection mechanism (kv_lora_rank/q_lora_rank) exists in the code or config; attention uses standard per-head q/k/v projections.

Sliding-window attention in its codeadopted

SWA layers (marked by hybrid_layer_pattern==1) use sliding_window=128 to build a sliding-window causal mask.

show evidencesliding_window = 128
self.sliding_window = getattr(config, "sliding_window", None) if is_swa else None
modeling_mimo_v2.py · L257
Interleaved sliding-window and global attention in its codeadopted

hybrid_layer_pattern interleaves full-attention layers (0) with sliding-window layers (1) in a fixed repeating pattern across 70 layers.

show evidence
is_swa_layer = config.hybrid_layer_pattern[layer_idx] == 1
        self.attention_type = "sliding_window_attention" if is_swa_layer else "full_attention"
modeling_mimo_v2.py · L395–396
Indexer-selected sparse attention (DeepSeek Sparse Attention) not in its codenot stated

No indexer-scoring or top-k token selection mechanism appears in the code or config (no index_topk/index_n_heads keys).

Linear-attention or state-space layers alongside full attention not in its codenot stated

All decoder layers use MiMoV2Attention (softmax attention, either full or sliding-window); there is no linear-attention or SSM mixer layer…

show evidence

All decoder layers use MiMoV2Attention (softmax attention, either full or sliding-window); there is no linear-attention or SSM mixer layer type.

self.self_attn = MiMoV2Attention(
            config, is_swa_layer, layer_idx, projection_layout=attention_projection_layout
        )
modeling_mimo_v2.py · L397–399
Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent mixer implementation or config keys (linear_num_key_heads, linear_conv_kernel_dim) are present.

Mamba-2 layers not in its codenot stated

No Mamba-2 SSD state-space block implementation or related config keys (mamba_num_heads, ssm_state_size) are present.

Learnable attention sink in its codeadopted

Sliding-window layers add a learned per-head attention_sink_bias as an extra logit column in the softmax denominator (enabled via…

show evidence

Sliding-window layers add a learned per-head attention_sink_bias as an extra logit column in the softmax denominator (enabled via add_swa_attention_sink_bias=true).

add_swa_attention_sink_bias = trueadd_full_attention_sink_bias = false
if sinks is not None:
        sinks = module.attention_sink_bias.reshape(1, -1, 1, 1).expand(query.shape[0], -1, query.shape[-2], -1)
        attn_weights = torch.cat([attn_weights, sinks], dim=-1)
modeling_mimo_v2.py · L84–86
self.attention_sink_bias = (
            nn.Parameter(torch.empty(self.num_attention_heads), requires_grad=False)
            if (
                (getattr(config, "add_full_attention_sink_bias", False) and not is_swa)
                or (getattr(config, "add_swa_attention_sink_bias", False) and is_swa)
            )
modeling_mimo_v2.py · L263–268
Gated attention output not in its codenot stated

No sigmoid gating multiplication is applied to the attention output before o_proj; attn_output goes directly into o_proj.

show evidence
attn_output = attn_output.reshape(*input_shape, -1).contiguous()
        attn_output = self.o_proj(attn_output)
modeling_mimo_v2.py · L349–350
QK normalization not in its codenot stated

No RMSNorm/LayerNorm is applied to query or key states before the attention dot product in MiMoV2Attention; queries/keys go straight from…

show the whole verdict

No RMSNorm/LayerNorm is applied to query or key states before the attention dot product in MiMoV2Attention; queries/keys go straight from projection/RoPE into matmul.

Partial RoPE code only in its codenot stated

Only rope_dim = head_dim * partial_rotary_factor (0.334) of each head is rotated, the remainder (query_nope/key_nope) carries no position…

show evidence

Only rope_dim = head_dim * partial_rotary_factor (0.334) of each head is rotated, the remainder (query_nope/key_nope) carries no position signal.

partial_rotary_factor = 0.334
query_rope, query_nope = query_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
        key_rope, key_nope = key_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
modeling_mimo_v2.py · L301–302
YaRN RoPE scaling not in its codenot stated

rope_scaling is not set in this config (no key present) so no YaRN scaling is applied; rotary embedding falls back to the default type.

show evidence
self.rope_type = "default"
modeling_mimo_v2.py · L448
Layers without positional encoding (NoPE) not in its codenot stated

All softmax-attention layers apply the same partial_rotary_factor RoPE scheme uniformly (full-attention and sliding-window layers both get…

show evidence

All softmax-attention layers apply the same partial_rotary_factor RoPE scheme uniformly (full-attention and sliding-window layers both get rotary embeddings via rotary_emb/swa_rotary_emb); there is no layer subset entirely without positional encoding.

position_embeddings = self.rotary_emb(hidden_states, position_ids)
        swa_position_embeddings = self.swa_rotary_emb(hidden_states, position_ids)
modeling_mimo_v2.py · L589–590
Mixture of experts in its codeadopted

Feed-forward layers route tokens to 8-of-384 experts chosen by a learned sigmoid router (noaux_tc top-k selection).

show evidencen_routed_experts = 384num_experts_per_tok = 8
self.experts = nn.ModuleList(
            [MiMoV2MLP(config, intermediate_size=config.moe_intermediate_size) for _ in range(config.n_routed_experts)]
        )
modeling_mimo_v2.py · L186–188
Shared expert not in its codenot stated

n_shared_experts is null in this config and no shared-expert module is instantiated in MiMoV2MoE.

show evidencen_shared_experts = null
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The MoE gate adds e_score_correction_bias to routing scores only for top-k selection (topk_method noaux_tc), not for weighting.

show evidencetopk_method = "noaux_tc"
scores_for_choice = scores.view(bsz * seq_len, -1) + self.e_score_correction_bias.unsqueeze(0)
modeling_mimo_v2.py · L159
Multi-token prediction layers code unclearadopted

The code ignores model.mtp.* weights on load and the config has no nextn/mtp layer count key, so it cannot be determined whether MTP…

show evidence

The code ignores model.mtp.* weights on load and the config has no nextn/mtp layer count key, so it cannot be determined whether MTP modules are declared for this checkpoint.

r"model\.mtp\..*",
modeling_mimo_v2.py · L623
SwiGLU feed-forward code only in its codenot stated

MLP and expert feed-forwards compute silu(gate_proj(x)) * up_proj(x) then down_proj, with hidden_act set to silu.

show evidencehidden_act = "silu"
return self.down_proj(self.act_fn(self.gate_proj(hidden_states)) * self.up_proj(hidden_states))
modeling_mimo_v2.py · L127

token mixer 5

softmax attention 4

global attention 1

Global attention used

“Interleaves Sliding Window Attention (SWA) and Global Attention (GA) with a 6:1 ratio”

sliding window attention 2

Sliding Window Attention used

“Interleaves Sliding Window Attention (SWA) and Global Attention (GA) with a 6:1 ratio”

Hybrid Sliding Window Attention used

“Its language backbone inherits from MiMo-V2-Flash's hybrid sliding-window attention architecture”

attention sink 1

Learnable attention sink bias used

“via learnable attention sink bias”

hybrid layer stacking 1

Hybrid Sliding-Window and Global Attention core

“Hybrid Attention Architecture: Interleaves Sliding Window Attention (SWA) and Global Attention (GA) with a 6:1 ratio and 128 sliding window.”

channel mixer 1

mixture of experts 1

Mixture of experts core

“MiMo-V2.5-Pro is a Mixture of Experts (MoE) model with 1.0 trillion total parameters, but only 42 billion active parameters are used…”

prediction head 2

Multi-Token Prediction used

“Multi-Token Prediction (MTP): Equipped with three lightweight MTP modules using dense FFNs.”

MTP speculative decoding used

“With one MTP layer for speculative decoding, the acceptance rate is about 90%.”

multimodal architecture 5

Native multimodal architecture core

“Native multimodal in one model. V2-Pro was text-and-code only. Multimodal capability existed in the separate V2-Omni model... V2.5…”

Native omnimodal architecture core

“MiMo-V2.5 is natively omnimodal — trained from scratch to see, hear, and act across modalities”

Native visual and audio understanding core

“With native visual and audio understanding, MiMo-V2.5 reasons seamlessly across modalities”

Dedicated visual and audio encoders with lightweight projectors used

“augmented with dedicated visual and audio encoders (both pretrained in-house) connected through lightweight projectors”

Projector warmup used

“projector warmup to align the audio and visual projectors with the language model”

context capacity 2

1M-token context window used

“supports up to 1 million tokens of context”

Progressive context extension used

“during which the context window is progressively extended from 32K to 256K to 1M”

language modelling objective 1

Text pre-training used

“Training goes through five stages: text pre-training on diverse corpora”

multi-token prediction objective 1

Multi-Token Prediction used

“We support MTP in vLLM and enhance the robustness of the inference engine in the RL system.”

training precision 1

FP8 mixed-precision training used

“Trained on 27T tokens using FP8 mixed precision”

data filtering 1

Multi-dimensional data filtering used

“applying multi-dimensional data filtering to increase reasoning pattern density in pre-training data”

data mixture & curriculum 1

Three-stage data mixture strategy used

“We adopt a three-stage data mixture strategy for pre-training.”

supervised fine-tuning 2

Supervised fine-tuning used

“Supervised Fine-Tuning (SFT) to build strong, foundational instruction-following skills”

Cold-started supervised fine-tuning model used

“we perform RL training on a cold-started SFT model, resulting in MiMo-7B-RL”

reinforcement learning algorithm 1

Reinforcement learning used

“RL and MOPD”

reward modelling 2

Rule-based verifiers used

“Each problem undergoes careful cleaning and difficulty assessment to ensure quality. We employ only rule-based accuracy rewards to avoid…”

Test-difficulty-driven code reward used

“To mitigate the sparse reward issue for challenging code problems, we introduce a test difficulty driven code reward.”

policy distillation 1

Multi-Teacher On-Policy Distillation core

“Multi-Teacher On-Policy Distillation (MOPD)”

rollout & RL infrastructure 3

Seamless Rollout Engine used

“We develop a Seamless Rollout Engine to accelerate RL training and validation.”

Continuous rollout with asynchronous rewards and early termination used

“Our design integrates continuous rollout, asynchronous reward computation, and early termination to minimize GPU idle time”

Data re-sampling strategy used

“We implement a data re-sampling strategy for easy problems to enhance rollout sampling efficiency and stabilize policy updates…”

agentic post-training 2

Agentic reinforcement learning used

“large-scale agentic RL”

Agentic post-training used

“supervised fine-tuning and agentic post-training”

decoding strategy 3

Multi-Token Prediction used

“We support MTP in vLLM and enhance the robustness of the inference engine in the RL system.”

EAGLE optional

“--speculative-algorithm EAGLE --speculative-num-steps 1 --speculative-eagle-topk 1”

Multi-layer EAGLE optional

“--enable-multi-layer-eagle”

serving parallelism 1

DeepEP optional

“--moe-a2a-backend deepep”

agent product 3

OpenCode optional

“Xiaomi explicitly positions it as a drop-in backend for scaffolds including Claude Code, OpenCode, and Kilo”

Claude Code optional

“Xiaomi explicitly positions it as a drop-in backend for scaffolds including Claude Code, OpenCode, and Kilo”

Kilo optional

“Xiaomi explicitly positions it as a drop-in backend for scaffolds including Claude Code, OpenCode, and Kilo”

unfiled 4

agentic model core

“MiMo-V2.5-Pro is Xiaomi’s most capable model to date, delivering significant improvements over its predecessor, MiMo-V2-Pro, in general…”

harness awareness core

“it makes full use of the affordances of its harness environment, manages its memory, and shapes how its own context is populated toward…”

multimodal pre-training used

“multimodal pre-training at scale on high-quality cross-modal data”

ngspice simulation used

“Wired into an ngspice simulation loop, in about an hour of closed-loop iteration — calling the simulator, reading waveforms, tweaking…”

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
mimo.xiaomi.com/mimo-v2-5official blogXiaomi MiMoread
huggingface.co/XiaomiMiMo/MiMo-V2.5-Promodel cardXiaomi MiMoread
github.com/XiaomiMiMo/MiMocode repoXiaomi MiMoread
openrouter.ai/xiaomi/mimo-v2.5-provendor docsOpenRouterread
artificialanalysis.ai/models/mimo-v2-5-prothird party analysisArtificial Analysisread
marktechpost.com/2026/04/22/xiaomi-releases-mimo-v2-5-pro-and-mimo-v2-5…newsMarkTechPostread
buildfastwithai.com/blogs/xiaomi-mimo-v2-5-pro-review-2026third party analysisBuildFastWithAIread
↑