Xiaomi
Xiaomi-Robotics-1
What Xiaomi-Robotics-1’s own documents say it is built from — every method with the sentence that describes it.
- Attention
- Qwen3-VL backbone
- Input
- text+image -> action
- Licence
- Apache-2.0
- Released
- 2026-07-28
Against the consensus recipe
Its documents state 0 of the 18 methods the field agrees on.
Data curation
- Benchmark decontaminationnot in its documents
- Synthetic data generationnot in its documents
- Corpus deduplicationnot in its documents
Training objective
- Multi-Token Predictionnot in its documents
- Knowledge insulationnot in its documents
- Quantization-aware distillationnot in its documents
Model architecture
- Mixture of expertsnot in its documents
- Multi-Token Predictionnot in its documents
- Grouped-query attentionnot in its documents
Optimization
- Muonnot in its documents
- Expert parallelismnot in its documents
- Quantization-Aware Trainingnot in its documents
Post-training
- Supervised fine-tuningnot in its documents
- Asynchronous reinforcement learningnot in its documents
- Multi-Teacher On-Policy Distillationnot in its documents
Inference & serving
- Thinking modenot in its documents
- Multi-Token Predictionnot in its documents
- FP8not in its documents
Checked in its code
Its code shows 3 of the 19 architecture features checked — 3 of them are not stated in its documents.
Sources say what a lab chose to describe; the modeling code says what the checkpoint runs. Each feature below was put to a code verifier, and only the config values and source lines it cited that were found in the files are shown, linked to the line.
Read from the model repository’s own code (modeling_mibot.py, configuration_mibot.py) and config.json at revision 3a6d0293bf, by anthropic/claude-sonnet-5 on 2026-11-07.
| Feature | Code | Its documents | Verdict |
|---|---|---|---|
| Grouped-query attention code only | in its code | not stated | The Qwen3VL text attention uses 32 query heads with only 8 key/value heads, repeated via repeat_kv for grouped-query attention. show evidencevlm_config.text_config.num_attention_heads = 32vlm_config.text_config.num_key_value_heads = 8 |
| Multi-head latent attention | not in its code | not stated | No low-rank KV latent projection (kv_lora_rank/q_lora_rank) exists anywhere in the attention code or config; attention uses standard… show the whole verdictNo low-rank KV latent projection (kv_lora_rank/q_lora_rank) exists anywhere in the attention code or config; attention uses standard per-head q/k/v projections. |
| Sliding-window attention | not in its code | not stated | No sliding_window config key or windowed-mask code is present for the language model; attention is full causal attention. |
| Interleaved sliding-window and global attention | not in its code | not stated | There is no layer_types or sliding_window_pattern config, and no code that interleaves local and global attention layers. |
| Indexer-selected sparse attention (DeepSeek Sparse Attention) | not in its code | not stated | No indexer scoring mechanism or index_topk/index_n_heads/index_head_dim config exists in this model. |
| Linear-attention or state-space layers alongside full attention | not in its code | not stated | All decoder layers use the same Qwen3VLTextAttention softmax attention; no linear-attention or SSM layer variants exist. |
| Gated DeltaNet layers | not in its code | not stated | No gated delta-rule recurrent mixer or linear_num_key_heads/linear_conv_kernel_dim config is present in the code. |
| Mamba-2 layers | not in its code | not stated | No Mamba-2 SSD blocks or ssm_state_size/mamba_num_heads config exist; the model is a standard transformer decoder. |
| Learnable attention sink | not in its code | not stated | The DiT policy head has a learned 'sink' embedding concatenated as a token, not a per-head sink logit added to the attention softmax… show the whole verdictThe DiT policy head has a learned 'sink' embedding concatenated as a token, not a per-head sink logit added to the attention softmax denominator; the language model's attention has no sink logit mechanism at all. |
| Gated attention output | not in its code | not stated | Neither the Qwen3VL text attention nor the DiT attention multiplies the attention output by an input-dependent sigmoid gate before the… show the whole verdictNeither the Qwen3VL text attention nor the DiT attention multiplies the attention output by an input-dependent sigmoid gate before the output projection; AdaLN gates (gate_msa/gate_mlp) scale the residual branch output, not an attention-internal gate, and apply to the action DiT head, not the language model. |
| QK normalization code only | in its code | not stated | Qwen3VLTextAttention applies RMSNorm to queries and keys per head before rotary embedding and the attention dot product. show evidence |
| Partial RoPE | not in its code | not stated | RoPE is applied to the full head dimension via rotate_half over the whole head_dim with no partial_rotary_factor or split dimension; no… show evidenceRoPE is applied to the full head dimension via rotate_half over the whole head_dim with no partial_rotary_factor or split dimension; no mechanism reduces the rotated portion. |
| YaRN RoPE scaling | not in its code | not stated | rope_scaling in text_config sets rope_type to 'default' with mrope_section/mrope_interleaved for multimodal RoPE, not YaRN, so YaRN… show evidencerope_scaling in text_config sets rope_type to 'default' with mrope_section/mrope_interleaved for multimodal RoPE, not YaRN, so YaRN scaling is not enabled. vlm_config.text_config.rope_scaling = {"mrope_interleaved": true, "mrope_section": [24, 20, 20], "rope_type": "defa… |
| Layers without positional encoding (NoPE) | not in its code | not stated | Every Qwen3VLTextDecoderLayer uses the same attention class with RoPE applied uniformly; there is no per-layer switch omitting positional… show the whole verdictEvery Qwen3VLTextDecoderLayer uses the same attention class with RoPE applied uniformly; there is no per-layer switch omitting positional encoding. |
| Mixture of experts | not in its code | not stated | No router, num_experts, or expert MLP module exists; Qwen3VLTextMLP is a single dense SwiGLU MLP per layer. |
| Shared expert | not in its code | not stated | There is no MoE structure at all, so no shared-expert mechanism or related config keys exist. |
| Auxiliary-loss-free load balancing (selection bias) | not in its code | not stated | No router or e_score_correction_bias exists since the model has no mixture-of-experts feed-forward layers. |
| Multi-token prediction layers | not in its code | not stated | No num_nextn_predict_layers or MTP module declarations exist in the config or code; the model only outputs a single next-token/action… show the whole verdictNo num_nextn_predict_layers or MTP module declarations exist in the config or code; the model only outputs a single next-token/action prediction head. |
| SwiGLU feed-forward code only | in its code | not stated | Qwen3VLTextMLP computes silu(gate_proj(x)) * up_proj(x) then down_proj, a SwiGLU feed-forward, matching hidden_act 'silu' in the text… show evidenceQwen3VLTextMLP computes silu(gate_proj(x)) * up_proj(x) then down_proj, a SwiGLU feed-forward, matching hidden_act 'silu' in the text config. vlm_config.text_config.hidden_act = "silu" |
filed at the root 1
“Xiaomi-Robotics-1 adopts a Mixture-of-Transformers (MoT) architecture consisting of a pre-trained vision-language model (VLM) (i.e.…”
token mixer 3
softmax attention 3
“To account for the temporal relationship between actions at different timesteps, we use causal attention in the DiT.”
“we change the original causal attention mask of the DiT to a -shape attention mask [20, 71, 16]”
attention sink 1
“We add a learnable attention sink token at the front of the state and noisy action tokens to stabilize the attention distribution during…”
positional encoding 1
“We first simply add an offset to the RoPE positional indices of the noisy action tokens to enable the model to distinguish tokens of noisy…”
normalization & residual 1
“We leverage adaptive normalization layers (adaLN) [49, 48] to inject the flow-matching timestep condition into the DiT for action…”
multimodal architecture 1
“we exclude these action-related tokens from the DiT’s attention computation, constraining the DiT tokens to attend solely to the…”
filed at the root 1
“Action prediction is supervised via a winner-takes-all scheme: only the candidate with the lowest distance is updated via backpropagation.”
language modelling objective 1
“We leverage chain-of-thought (CoT) labeling as in ERVLA [61] and train our model with a 50% probability on the next-token-prediction loss…”
auxiliary loss 3
“we sample from a Beta distribution, placing more weight on noisier timesteps during training.”
“we introduce an auxiliary action-generation supervision on the VLM.”
“We mask out the dimensions corresponding to missing action components during loss computation.”
training runtime 1
“We fine-tune Xiaomi-Robotics-1 on the data of all four tasks using the asynchronous training recipe proposed in [8].”
data filtering 1
“We filter out idle segments within trajectories to prevent the model from learning uninformative or noisy signals.”
synthetic data 5
“we develop an auto-labeling pipeline that first divides each trajectory into equal-length segments and leverage Qwen3.5-27B [70] to…”
“For visual grounding, we develop a rigorous cross-validated consensus mechanism that integrates Grounded SAM [55], Grounding DINO 1.5…”
“We leverage Qwen3.5 [70] to annotate human-segmented video clips with language instructions.”
“VQA and captioning annotation quality is further refined through re-labeling using state-of-the-art pre-trained VLMs [1].”
“For embodied reasoning & planning, we leverage pre-trained VLMs to generate data from root trajectories”
data mixture & curriculum 3
“We sample vision-language data and robot trajectory data at a ratio of 1:6.”
“we co-train the model with the entire robot trajectory data and vision-language data”
“we further co-train the model on a high-quality vision-language dataset curated in our previous work [8] under the next-token prediction…”
sequence packing 1
“we pack all vision-language tokens within a batch into a single sequence for a VLM forward pass.”
filed at the root 1
“We adopt a two-stage training recipe consisting of pre-training and post-training.”
inference scheduling 2
“Asynchronous execution enables the robot to continue rolling out trajectories during model inference.”
“During deployment, we carefully align the timesteps of action chunks generated from consecutive inferences to ensure continuous and…”
inference kernel 1
“attn_implementation="flash_attention_2",”
context management 1
“observation history: 4”
unfiled 18
“the DiT generates the action chunk via flow-matching [49]”
“The model is optimized to predict actions by jointly minimizing the flow-matching loss of the DiT and the regression loss of the VLM…”
“To account for the multi-modality in trajectories, we adopt Choice Policies [51] for action prediction.”
“we leverage Choice Policies [59] to enable action generation directly within the VLM framework [8].”
“we unify the orientation of the end-effector frames across all robot data and UMI data in both the pre-training and post-training datasets.”
“For reproducible multi-GPU evaluation, use the standard eight-server launcher in the companion source repository.”
“The clean action chunk is recovered via a 5-step Euler integration”
“first divides each trajectory into equal-length segments”
“we amortize its cost by sampling four flow-matching timesteps per sample.”
“The custom model and processor must be loaded with trust_remote_code=True.”
“we dynamically re-weight the flow-matching loss based on the error between the online-predicted actions and the ground-truth actions.”
“actions per query: 16”
“We condition the next inference by prefixing the noisy actions with the actions from step to step of the current chunk.”
“we develop a producer–consumer pipeline that decouples clip segmentations from caption labeling: while CPU worker threads cut per-segment…”
“For arm actions, we adopt relative delta end-effector (EE) poses with respect to the current state”
“observation interval: 2”
“we synchronize all input modalities by resampling them to a unified 30Hz timeline using timestamps.”
“we follow training RTC [5] and condition action generation on previously committed actions by prefixing them to the noisy action tokens in…”
Sources
The curated source list for this model. "not fetched" means the URL is recorded but its text was not read in the current run.
| Source | Kind | Publisher | Status |
|---|---|---|---|
| huggingface.co/XiaomiRobotics/Xiaomi-Robotics-1-RoboCasa365 | model card | Xiaomi | read |
| arxiv.org/abs/2607.15330 | technical report | Xiaomi | read |
| arxiv.org/abs/2602.12684 | technical report | Xiaomi | read |