Open Models Almanac
ModelsMiMo-V2.6-Flash

Xiaomi · usage rank #10

MiMo-V2.6-Flash

What MiMo-V2.6-Flash’s own documents say it is built from — every method with the sentence that describes it.

Parameters
309B / 15B active
Context
1M
Attention
sliding/global hybrid39 SWA (window 128) : 9 global GQA layers, attention sink
Experts
256 routed, top-8, no shared
Input
text, image, audio, video
Licence
MIT
Released
2026-09-22

Rank by open-weight tokens on OpenRouter, week of 2026-10-26.

Against the consensus recipe

Its documents state 3 of the 18 methods the field agrees on.

The recipe from the overview: the methods the most labs adopt, per stage. A missing one is something its documents do not say, not something the model lacks.
  1. Data curation

  2. Training objective

  3. Model architecture

  4. Optimization

  5. Post-training

  6. Inference & serving

Checked in its code

Its code shows 9 of the 19 architecture features checked — 6 of them are not stated in its documents.

Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.

Read from the model repository’s own code (modeling_mimo_v2.py, configuration_mimo_v2.py) and config.json at revision 5711b26816, by anthropic/claude-sonnet-5 on 2026-09-25.

FeatureCodeIts documentsVerdict
Grouped-query attention code only in its codenot stated

Full-attention layers use 64 query heads with 4 KV heads (and SWA layers use 64 query heads with 8 KV heads), repeated via repeat_kv…

show evidence

Full-attention layers use 64 query heads with 4 KV heads (and SWA layers use 64 query heads with 8 KV heads), repeated via repeat_kv, giving grouped-query attention throughout.

num_attention_heads = 64num_key_value_heads = 4swa_num_key_value_heads = 8
self.num_key_value_groups = self.num_attention_heads // self.num_key_value_heads
modeling_mimo_v2.py · L259
key_states = repeat_kv(key, module.num_key_value_groups)
    value_states = repeat_kv(value, module.num_key_value_groups)
modeling_mimo_v2.py · L82–83
Multi-head latent attention not in its codenot stated

There is no low-rank KV latent projection (no kv_lora_rank/q_lora_rank in config or code); attention uses standard per-head q/k/v…

show the whole verdict

There is no low-rank KV latent projection (no kv_lora_rank/q_lora_rank in config or code); attention uses standard per-head q/k/v projections with GQA instead.

Sliding-window attention code only in its codenot stated

Layers flagged 1 in hybrid_layer_pattern are is_swa layers whose sliding_window is set to the non-null config value 128, and a…

show evidence

Layers flagged 1 in hybrid_layer_pattern are is_swa layers whose sliding_window is set to the non-null config value 128, and a sliding-window causal mask is built for them.

sliding_window = 128hybrid_layer_pattern = [0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1…
self.sliding_window = getattr(config, "sliding_window", None) if is_swa else None
modeling_mimo_v2.py · L262
causal_mask_mapping["sliding_window_attention"] = create_sliding_window_causal_mask(**mask_kwargs)
modeling_mimo_v2.py · L1657
Interleaved sliding-window and global attention in its codeadopted

hybrid_layer_pattern interleaves full-attention layers (0) with sliding-window layers (1) in a fixed repeating pattern across the 48…

show evidence

hybrid_layer_pattern interleaves full-attention layers (0) with sliding-window layers (1) in a fixed repeating pattern across the 48 decoder layers.

hybrid_layer_pattern = [0, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1…
is_swa_layer = config.hybrid_layer_pattern[layer_idx] == 1
        self.attention_type = "sliding_window_attention" if is_swa_layer else "full_attention"
modeling_mimo_v2.py · L400–401
Indexer-selected sparse attention (DeepSeek Sparse Attention) not in its codenot stated

No indexer scoring or top-k token selection mechanism exists in the code, and no index_topk/index_n_heads config keys are present.

Linear-attention or state-space layers alongside full attention not in its codenot stated

All token-mixing layers in this file are softmax attention (full or sliding-window); there is no linear-attention, Mamba, or state-space…

show the whole verdict

All token-mixing layers in this file are softmax attention (full or sliding-window); there is no linear-attention, Mamba, or state-space mixer implementation.

Gated DeltaNet layers not in its codenot stated

No gated delta-rule recurrent mixer is implemented; only softmax attention layers exist.

Mamba-2 layers not in its codenot stated

No Mamba-2/SSD selective state-space block is implemented anywhere in the given files.

Learnable attention sink code only in its codenot stated

Language-model attention has a learned per-head attention_sink_bias parameter appended as an extra logit column in eager attention…

show evidence

Language-model attention has a learned per-head attention_sink_bias parameter appended as an extra logit column in eager attention softmax, enabled for SWA layers by add_swa_attention_sink_bias=true.

add_swa_attention_sink_bias = trueadd_full_attention_sink_bias = false
if sinks is not None:
        sinks = module.attention_sink_bias.reshape(1, -1, 1, 1).expand(query.shape[0], -1, query.shape[-2], -1)
        attn_weights = torch.cat([attn_weights, sinks], dim=-1)
modeling_mimo_v2.py · L89–91
self.attention_sink_bias = (
            nn.Parameter(torch.empty(self.num_attention_heads), requires_grad=False)
            if (
                (getattr(config, "add_full_attention_sink_bias", False) and not is_swa)
                or (getattr(config, "add_swa_attention_sink_bias", False) and is_swa)
            )
            else None
        )
modeling_mimo_v2.py · L268–275
Gated attention output not in its codenot stated

The language-model attention output goes straight through o_proj with no sigmoid gate multiplying it; no attn_output_gate/use_output_gate…

show the whole verdict

The language-model attention output goes straight through o_proj with no sigmoid gate multiplying it; no attn_output_gate/use_output_gate config or code exists for the LM.

QK normalization not in its codenot stated

Query and key tensors are used directly for RoPE and attention without any RMSNorm/LayerNorm normalization step in MiMoV2Attention.

Partial RoPE code only in its codenot stated

Each head's dimension is split into a rotary part (rope_dim, sized by partial_rotary_factor 0.334) and a non-rotary part (head_dim…

show evidence

Each head's dimension is split into a rotary part (rope_dim, sized by partial_rotary_factor 0.334) and a non-rotary part (head_dim - rope_dim) that carries no position signal.

partial_rotary_factor = 0.334head_dim = 192
self.rope_dim = int(self.head_dim * getattr(config, "partial_rotary_factor", 1.0))
modeling_mimo_v2.py · L253
query_rope, query_nope = query_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
        key_rope, key_nope = key_states.split([self.rope_dim, self.head_dim - self.rope_dim], dim=-1)
modeling_mimo_v2.py · L306–307
YaRN RoPE scaling not in its codenot stated

rope_parameters/rope_type is set to "default" in this config, not YaRN, so no YaRN scaling is applied.

show evidencerope_parameters.rope_type = "default"rope_parameters.type = "default"
Layers without positional encoding (NoPE) not in its codenot stated

Every attention layer (full or sliding-window) applies rotary embeddings via its own rotary embedding module; there is no layer that skips…

show the whole verdict

Every attention layer (full or sliding-window) applies rotary embeddings via its own rotary embedding module; there is no layer that skips RoPE entirely for softmax attention.

Mixture of experts in its codeadopted

MoE feed-forward layers route each token via a sigmoid-scored top-8-of-256 router (MiMoV2MoEGate/MiMoV2MoE) applied on layers flagged in…

show evidence

MoE feed-forward layers route each token via a sigmoid-scored top-8-of-256 router (MiMoV2MoEGate/MiMoV2MoE) applied on layers flagged in moe_layer_freq.

n_routed_experts = 256num_experts_per_tok = 8
self.experts = nn.ModuleList(
            [MiMoV2MLP(config, intermediate_size=config.moe_intermediate_size) for _ in range(config.n_routed_experts)]
        )
modeling_mimo_v2.py · L191–193
_, topk_idx = torch.topk(tmp_scores, k=self.top_k, dim=-1, sorted=False)
            topk_weight = scores.gather(1, topk_idx)
modeling_mimo_v2.py · L175–176
Shared expert not in its codenot stated

n_shared_experts is null in this config, and MiMoV2MoE builds no additional always-on expert module.

show evidencen_shared_experts = null
Auxiliary-loss-free load balancing (selection bias) code only in its codenot stated

The router adds e_score_correction_bias to scores only for top-k selection (noaux_tc topk_method), while topk_weight for weighting is…

show evidence

The router adds e_score_correction_bias to scores only for top-k selection (noaux_tc topk_method), while topk_weight for weighting is gathered from the unbiased sigmoid scores.

topk_method = "noaux_tc"
scores_for_choice = scores.view(bsz * seq_len, -1) + self.e_score_correction_bias.unsqueeze(0)
modeling_mimo_v2.py · L164
_, topk_idx = torch.topk(tmp_scores, k=self.top_k, dim=-1, sorted=False)
            topk_weight = scores.gather(1, topk_idx)
modeling_mimo_v2.py · L175–176
Multi-token prediction layers in its codeadopted

config declares num_nextn_predict_layers=3 for MTP modules, even though the given modeling file does not build them (weights are ignored…

show evidence

config declares num_nextn_predict_layers=3 for MTP modules, even though the given modeling file does not build them (weights are ignored via the mtp key pattern), which counts per the check's rule for config-declared nonzero MTP counts.

num_nextn_predict_layers = 3
r"model\.mtp\..*",
modeling_mimo_v2.py · L1694
SwiGLU feed-forward code only in its codenot stated

MiMoV2MLP (used both as dense MLP and MoE expert) computes down_proj(act_fn(gate_proj(x)) * up_proj(x)) with hidden_act=silu, i.e. SwiGLU.

show evidencehidden_act = "silu"
return self.down_proj(self.act_fn(self.gate_proj(hidden_states)) * self.up_proj(hidden_states))
modeling_mimo_v2.py · L132

token mixer 4

softmax attention 2

sliding window attention 2

Alternating Row-Major and Column-Major Token Serialization core

“Local SWA layers alternate between row-major and column-major token serialization to support information propagation along both spatial…”

Sink-Augmented Sliding Window Attention core

“It replaces fixed, non-overlapping window attention in MiMo-VL-7B with sink-augmented SWA, enabling information exchange across window…”

hybrid layer stacking 2

Hybrid Sliding-Window and Global Attention core

“repeated hybrid blocks that interleave Local Sliding Window Attention (SWA) and Global Attention (GA)”

Hybrid sparse mixture-of-experts Transformer architecture core

“MiMo-V2.6 builds a hybrid sparse MoE Transformer backbone that interleaves Local Sliding Window Attention (SWA) with Global Attention (GA)”

channel mixer 2

mixture of experts 2

Mixture of experts core

“Built on a Mixture-of-Experts architecture with 309B total parameters and 15B activated per token”

shared experts 1

MoE without shared experts core

“both use sparse MoE FFNs without shared experts.”

prediction head 1

Multi-Token Prediction core

“Multi-Token Prediction (MTP): 5-layer speculative decoder”

multimodal architecture 6

Four-frame audio patches with within-patch bidirectional attention core

“Every four consecutive frames are grouped into an audio patch and processed by a Transformer with bidirectional self-attention confined to…”

Joint multimodal input understanding core

“Joint input and understanding of images, video, audio, and text”

Two-stage audio encoding with tokenization and patch encoding core

“Audio encoding in MiMo-V2.6 consists of two stages: audio tokenization with the Audio Tokenizer, followed by patch encoding.”

Data-parallel-first multimodal encoding used

“encoding runs data-parallel first—image items are balanced across ranks independently of where each sequence’s tokens land”

Joint full-model multimodal training used

“In the second stage, we integrate the backbone with our in-house pre-trained ViT and audio encoder and jointly train the full model on…”

Two-stage text-then-multimodal pre-training used

“We adopt a two-stage pre-training strategy. In the first stage, we train the language backbone on text-only data”

context capacity 1

1M-token context window core

“1M-token context window, suited for mixed-modality input and multi-agent collaboration”

language modelling objective 1

Cross-entropy objective used

“optimize the resulting VLM solely on multimodal understanding data using a cross-entropy objective”

auxiliary loss 1

Loss masking of flagged trajectory content used

“mask excludes the hit content from the loss”

filed at the root 1

Fused loss computation used

“All loss computation is fused into one kernel: the policy-gradient loss (with or without top-prenormalization) and the OPD loss…”

optimizer 1

Muown used

“We therefore switch to a variant of Muon optimizer, Muown, for the hidden weight matrices during mid-training”

quantization-aware training 2

Quantize-Dequantize Training core

“we apply quantize–dequantize (QDQ) to the experts after each parameter update.”

MXFP4 Quantization-Aware Training used

“We employ MXFP4 quantization-aware training (QAT) during mid-training, allowing the model to adapt to low-precision computation while…”

training stability 2

Behavioral regularization used

“To improve RL training stability and encourage efficient, reliable behavior, we introduce two complementary mechanisms.”

Safeguards against training drift and reward hacking used

“safeguards against training drift and reward hacking.”

training parallelism 1

Window-bounded KV exchange for sliding-window layers used

“the sliding-window layers exchange only the KV their queries can reach—at most a window-sized segment—so per-layer traffic is bounded by…”

training runtime 1

Keep optimizer states in CPU memory used

“Optimizer states stay in CPU memory and are copied back only for parameter updates.”

synthetic data 3

Automated large-scale data synthesis and environment construction used

“We therefore invest in large-scale environment synthesis and curation across coding, general professional workflows, visual artifacts, and…”

Multi-agent synthesis of local software mocks for reproducible environments used

“use an automated multi-agent workflow to build local software mocks that reproduce supported operations, state changes, output formats…”

Training SFT teachers on synthetic demonstrations for open-domain tasks used

“we train SFT teachers on high-quality synthetic demonstrations”

data mixture & curriculum 5

Sample Mixer core

“we implement the Sample Mixer, with four mechanisms filling the specified training distribution”

Agent-centric multimodal data mixture used

“we train the MiMo-V2.6 model series on a diverse, agent-centric data mixture”

Data Scheduler used

“we have maintained a Data Scheduler that targets a specified training distribution across data sources”

Dynamic sampling for mixed-task reinforcement learning used

“we have maintained a Data Scheduler that targets a specified training distribution across data sources, together with dynamic sampler and…”

Smooth weighted round-robin source scheduling used

“These weights drive smooth weighted round-robin”

sequence packing 1

Group-local sequence packing used

“one packer per training tensor-parallel (TP) group serves every rank in the group”

supervised fine-tuning 3

SFT checkpoint intended as an RL starting point core

“this checkpoint is an SFT starting point for RL research, not a finished assistant”

Self-correction cold start used

“Cold start from self-correction — the model reflects on and rewrites its own misaligned turns into grounded next steps.”

Supervised fine-tuning on MiMo-generated task data used

“We obtain the model by supervised fine-tuning Qwen3.5-9B (Qwen Team, 2025) on MiMo-generated data spanning coding, general-domain, visual…”

reinforcement learning algorithm 16

Asynchronous Group Relative Policy Optimization core

“Fully asynchronous Group Relative Policy Optimization (GRPO) on very large batches”

Groupwise Advantage Redistribution core

“Groupwise Advantage Redistribution (GAR) ranks passing trajectories online and moves advantage toward higher-quality solutions.”

Penalty Module core

“a configurable Penalty Module applies loss masking and advantage shaping”

Dropping zero-advantage rollout groups default

“groups whose advantages are all zero are dropped by default”

Dynamic sampling of informative rollout groups used

“we incorporate a dynamic sampler to filter groups that are all-pass or all-fail”

Rollout Routing Replay used

“Rollout Routing Replay (R3) (Ma et al., 2025) records the expert indices used during rollout and replays them during training, reproducing…”

Advantage shaping used

“advantage shaping sets, scales, or subtracts the advantages on hit tokens”

Domain-specific GRPO training used

“we conduct GRPO training separately for coding, cybersecurity, general-domain, and visual tasks using the corresponding released…”

Flagged-token advantage penalty and masking used

“This masks flagged tokens in positive trajectories and penalizes them more strongly in negative trajectories.”

Freezing the MoE router during reinforcement learning used

“We therefore freeze the router for RL training; the frozen-router run keeps all three statistics flat”

Group-relative length penalty used

“For each prompt q with G sampled rollouts, let R_i be the original outcome reward, P_q the indices of successful rollouts, and ℓ_i = |o_i|…”

Hierarchical penalty escalation used

“a context with no surviving model turns is dropped, a sequence with no surviving context receives zero advantage, and a sample with no…”

Quality-weighted advantage redistribution used

“quality factors first downweight lower-quality passes, after which a common factor redistributes the removed positive advantage mass among…”

Replay top-k/top-p sampling candidate sets during training used

“we record each token’s candidate set during rollout and renormalize the training log-probabilities within it, matching the sampler’s…”

Single-harness reinforcement learning used

“The coding results in Table 6 are obtained through single-harness RL.”

Zero-reward correction for externally supported trajectories used

“When evidence confirms dependence on an external or leaked answer, we reset the trajectory’s effective reward to zero and treat it as a…”

reward modelling 13

Groupwise Agentic Grading core

“we introduce groupwise agentic grading to provide more informative reward signals beyond binary test cases”

Groupwise Reward Synthesis core

“Groupwise Reward Synthesis (GRS) builds task-specific rubrics offline from contrasting rollouts and fuses rubric quality with test outcomes”

Adversarial-agent reward-hacking audit used

“A dedicated hack agent then probes the prepared environments for remaining leaks and exploitable weaknesses.”

Verifier cross-checking used

“Throughout RL, environment hardening, adversarial screening, and verifier cross-checks keep the loop honest against reward hacking.”

Behavior rubrics used

“Behavior rubrics describe observable practices that support reliable problem solving, such as gathering relevant evidence and checking the…”

Five-dimension comparative grading of passing patches used

“comparing passing patches along five dimensions: the suitability of the solution approach, precision in implementing that approach without…”

Monitoring-only penalty strategy used

“monitor records metrics alone”

Multiplicative reward synthesis from test and rubric scores used

“We synthesize the final training reward by multiplying this test reward by the two rubric scores”

Negative rubric checks for unintended side effects used

“To test resistance to reward hacking, we include negative checks for unintended changes to unrelated files or databases”

Offline trajectory auditing for reward hacking used

“We regularly audit agent trajectories offline for reward hacking throughout training.”

Rule-based or model-judged trajectory error detection used

“A Rule judges a segment, context, or sequence by handcrafted logic or a model-based judge.”

Segment-level behavioral penalties used

“motivating segment-level rules for format violations and tool call errors such as malformed markup, invalid tool names, or malformed…”

Solution-quality rubrics used

“Solution rubrics describe task-relevant properties of a good implementation, including satisfaction of the requirements, appropriate…”

policy distillation 6

Multi-Prefix Multi-Teacher On-Policy Distillation used

“After mixed RL, we use Multi-Prefix Multi-Teacher On-Policy Distillation (MOPD2) to combine capabilities from teachers trained for…”

Prefix-Conditioned On-Policy Distillation used

“SFT data provide prefix contexts rather than fixed continuation targets.”

Autonomous Student Rollouts used

“we retain autonomous student rollouts in domains with suitable mixRL teachers”

Distillation on MiMo-Generated Data used

“a Qwen3.5-9B fine-tune trained on MiMo-generated data”

Model Distillation used

“These progressive qualitative improvements align with the quantitative trajectory (61.7 → 64.0 → 72.4) observed in Table 6, jointly…”

Standard Multi-Teacher On-Policy Distillation used

“Standard MOPD uses RL teachers to supervise full student rollouts.”

rollout & RL infrastructure 25

Asynchronous reinforcement learning core

“we scale batch size and training throughput through fully asynchronous training, processing thousands of long-horizon rollouts and…”

Agent Loop core

“each sequence runs as an Agent Loop that owns the environment lifecycle, manages the dialogues it produces, and calls the inference engine…”

Agent-centric rollout execution core

“we adopt an agent-centric execution model and organize trajectory data as a hierarchy”

Hierarchical trajectory data organization core

“We organize the trajectory data into a four-level hierarchy: Sample → Sequence → Context → Segment.”

Payload Porter core

“the Payload Porter keeps the driver scheduling on lightweight metadata alone. Heavy payloads are written once to a distributed store and…”

Persistent multi-tenant harness pool core

“the Harness Pool hosts concurrent harness instances and Agent Loops in persistent multi-tenant actor pools”

Predictive Rollout Dispatch core

“We jointly estimate KV demand and expected inference concurrency to guide the admission and placement of new rollouts across ranks.”

Partial rollout used

“we have maintained a Data Scheduler that targets a specified training distribution across data sources, together with dynamic sampler and…”

Adaptive Rollout Concurrency used

“Slower sources need more concurrent rollouts to sustain the same training contribution.”

Isolated resettable sandboxes for agent rollouts used

“each rollout runs in an isolated sandbox that can be restored to a fixed initial state”

Adaptive Rollout Scheduling used

“Within these budgets, scheduling balances long-run generation demand with progress toward the current training batch.”

Background-thread offloading of blocking rollout work used

“blocking work—environment operations and tokenization—runs on background threads”

Composite early-stop strategy that zeroes reward and masks affected contexts used

“The composite early stop strategy halts the rollout as soon as its rule fires, zeroes the outcome reward, applies separate actions to the…”

Decoupled control plane and data plane used

“decouple the control plane from the data plane to buffer and transfer massive trajectories carrying routing and multi-modal payloads”

Dialogue prefix matching used

“Prefix matching locates the dialogue that an incoming request extends, and only the new suffix is tokenized and passed to the inference…”

Duration- and acceptance-adjusted per-source oversampling used

“We assign each source a scheduling budget of (1 + pᵢ)mᵢ groups, where the oversampling ratio pᵢ satisfies”

Heterogeneous Agent Harnesses used

“We configure harness codebases, agent behavior, and environment settings separately to accommodate diverse tasks within a single training…”

Incremental multimodal-delta transfer used

“During rollout, the Agent Loop therefore ships only the multi-modal delta between requests”

Mixed-task rollout infrastructure used

“These advances are supported by asynchronous training, mixed-task rollout infrastructure, training–inference consistency mechanisms”

Multimodal metadata/payload separation used

“Multi-modal payloads follow the same meta/payload split, but demand extra care”

Per-source priors for rollout length estimation used

“Per-source priors estimate a rollout’s total sequence length (input and generated tokens), summed across contexts”

Sample Mixer used

“A sample mixing mechanism, working together with dynamic sampling and partial rollout, stabilizes per-task sample composition in train…”

Sample replay used

“We use sample replay to accelerate initial collection from slow sources.”

Training–inference consistency mechanisms used

“These advances are supported by asynchronous training, mixed-task rollout infrastructure, training–inference consistency mechanisms”

Asynchronous group-wise grading optional

“A group-wise grader, when configured, runs fully asynchronously alongside the Agent Loops”

agentic post-training 6

Agentic reinforcement learning core

“a practical approach to advancing foundation models through large-scale agentic reinforcement learning.”

Environment hardening during reinforcement learning used

“Throughout RL, environment hardening, adversarial screening, and verifier cross-checks keep the loop honest against reward hacking.”

Container-level network isolation used

“We enforce container-level network isolation to restrict access to upstream fixes and alternative package versions that may contain the…”

Environment preparation to prevent solution leakage used

“We therefore remove build logs, verifier outputs, residual patches, and project-generated binaries or bytecode that could expose it.”

Multi-harness reinforcement learning used

“we jointly optimize the model across four mini-harnesses and evaluate it on these harnesses and three additional held-out harnesses.”

Unified trajectory representation for agentic RL used

“we design a unified trajectory representation and a penalty mechanism that together refine the learning signal”

mid-training & continual pretraining 2

Agent-centric mid-training used

“Building on an omni-capable foundation and agent-centric mid-training, we scale RL along three dimensions”

Mid-training alignment data from reward-hacking cases used

“For each case, MiMo reflects on the faulty reasoning, revises the relevant turn, and continues with actions grounded in the task…”

1 more mentioned or evaluated
Deficit-corrected scheduling evaluated

rollout & RL infrastructure “Deficit-corrected scheduling (α = 0.5) improves occupancy stability over Deficit-based scheduling (α = 0) and collection balance over…”

decoding strategy 6

DFlash core

“5-layer SWA MTP drafter (DFlash-style). Predicts 7 subsequent tokens per forward pass for parallel verification.”

Five-layer SWA MTP drafter core

“it predicts 7 subsequent tokens in a single forward pass for parallel verification by the backbone”

Distribution-matched speculative-draft fine-tuning used

“DFlash is then finetuned on early RL rollout logs resampled to match the RL training distribution.”

Throughput-based draft-block sizing used

“At large RL batch sizes, we select the draft block size based on end-to-end throughput rather than acceptance rate alone.”

Speculative decoding optional

“--speculative-algorithm EAGLE”

EAGLE optional

“--speculative-algorithm EAGLE”

KV cache management 3

Persistent per-dialogue-context KV caching core

“each dialogue context has a persistent key: within one policy version, later turns hit the cached KV—including generated tokens—and…”

Asynchronous KV-cache offloading used

“Offloading and restoration run on side CUDA streams rather than the compute stream, never stalling generation.”

Hierarchical KV caching across HBM and pinned host memory used

“a state resides in HBM during GPU time and in a pinned host pool during tool time.”

inference quantization 2

Dynamic activation scaling core

“quant_method: fp8 with e4m3 and dynamic activation scaling”

FP8 low-precision speculative draft computation used

“Low-precision draft computation (FP8) is also applied to reduce drafting overhead.”

serving parallelism 1

DeepEP used

“its SGLang example spans two nodes with --tp 16 --ep 16 and DeepEP for expert parallelism”

inference scheduling 1

Greedy feasible-rank placement used

“a greedy heuristic selects the feasible rank with the greatest remaining capacity”

agentic scaffolding 2

Minimal, modular mini-harnesses for controlled harness diversity used

“All mini-harnesses start from the same minimal agent loop, which already includes the components needed for task completion—system prompt…”

Request timeouts and retries used

“If you put Pro behind an agent loop, set a request timeout and a retry.”

1 more mentioned or evaluated
Multi-agent collaboration mentioned

agentic scaffolding “suited for mixed-modality input and multi-agent collaboration”

inference engine 1

vLLM tensor parallelism across eight GPUs used

“Xiaomi’s model card serves it with vLLM at --tensor-parallel-size 8”

filed at the root 2

Behavioral testing with stubs and edge cases used

“scripts were run against stubbed pg_dump and aws binaries with a missing-variable test, a real run, and a retention check using fake…”

Rollout-based audit used

“we further assess their alignment through rollout-based auditing. Each task is attempted four times by a coding agent.”

evaluation harness 1

Atomic binary rubric evaluation used

“Task completion is evaluated using atomic, binary rubric items: code-based checks verify deterministic properties”

human & real-world evaluation 1

Same-prompt comparative DevOps testing used

“a same-prompt DevOps test graded by kubeconform, shellcheck, tofu validate, a live k3s cluster, and a behaviour test for every bash script”

filed at the root 2

Artifact validation with linters and runtime tests used

“Every artifact went through its linter, then the part that matters: manifests were applied to k3s and curled through the Service, and…”

Training–inference consistency through routing and sampling alignment used

“We align the training and inference engines on MoE routing and top-p sampling candidate sets”

unfiled 8

MXFP4 weight storage with block size 32 core

“store_dtype: mxfp4 with a block size of 32”

Large collection of reinforcement-learning task environments used

“more than 7,000 RL task environments”

Live reinforcement-learning training used

“the final RL stage ran live for under six days”

Optimization on real agent-harness environments used

“Optimized on real harness environments—drop-in ready for agent harnesses like MiMo Desktop”

Pin API requests to the first-party provider and disable fallbacks used

“We pinned OpenRouter to the first-party Xiaomi endpoint with fallbacks disabled, so every call below hit Xiaomi’s servers and not a…”

Repeated test execution to detect flaky tests and reward instability used

“We check reward stability through repeated execution.”

Run ShellCheck at info severity to catch possible misspellings used

“Run ShellCheck at info level on anything a model writes.”

Use writable Kubernetes emptyDir volumes for nginx runtime directories used

“The fix is to give nginx writable emptyDir volumes at /var/cache/nginx and /var/run, which is what Pro’s first run did.”

Sources

The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.

SourceKindPublisherStatus
mimo.xiaomi.com/mimo-v2-6official blogXiaomi MiMonot fetched
huggingface.co/XiaomiMiMo/MiMo-V2.6-Pro-RL/resolve/main/MiMo_V2_6_techn…technical reportXiaomi MiMoread
huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RLmodel cardXiaomi MiMoread
mimo.mi.com/models/en-US/mimo-v2.6-flashvendor docsXiaomi MiMoread
openrouter.ai/xiaomi/mimo-v2.6-flashvendor docsOpenRouterread
computingforgeeks.com/xiaomi-mimo-v2-6-pro-flashthird party analysisComputingForGeeksread
↑