PublishersXiaomi LLM-Core

Xiaomi LLM-Core

1 document read from this publisher.

MiMo-V2-Flash Technical Report

technical report · 2026-01-06 · 39 techniques

Mixture of Experts Multi-Token Prediction Hybrid Attention Speculative decoding Group Relative Policy Optimization Multi-Teacher On-Policy Distillation Supervised fine-tuning Grouped-query attention Reinforcement Learning Sliding Window Attention On-Policy Distillation Cosine decay Hybrid Sliding Window Attention Learnable attention sink bias Partial rollout Rotary Position Embedding AdamW FP8 mixed-precision training Global attention No Shared Experts Rollout Routing Replay Data Scheduler RadixCache Seamless Rollout Engine Tool Manager Toolbox Bash commands for context retrieval Dense feed-forward network Discarding tool-call history Expert bias update factor Gradient clipping Load balancing Memory compression MoE sequence auxiliary loss MoE with 256 experts and top-8 routing Outcome Reward Model Request-level prefix cache Reverse KL divergence Staleness-aware truncated importance sampling