SourcesXiaomi LLM-Core
publisher · 1 source
Xiaomi LLM-Core
1 source read from this publisher, newest first, with the methods each one describes.
MiMo-V2-Flash Technical Report
technical report · 2026-01-06 · 38 methods
Mixture of expertsMulti-Token PredictionMulti-Teacher On-Policy DistillationSupervised fine-tuningHybrid Sliding-Window and Global AttentionMTP speculative decodingReinforcement learningGroup Relative Policy OptimizationSliding Window AttentionGrouped-query attentionAdamWOn-Policy DistillationGlobal-norm gradient clippingGlobal attentionRotary Position EmbeddingCosine decayHybrid Sliding Window AttentionPartial rolloutFP8 mixed-precision trainingLearnable attention sink biasMoE without shared expertsRollout Routing ReplayData SchedulerLoad balancingRadixCacheReverse KL divergenceSeamless Rollout EngineTool ManagerToolboxBash commands for context retrievalDense feed-forward networkDiscarding tool-call historyExpert bias update factorMemory compressionMoE sequence auxiliary lossOutcome Reward ModelRequest-level prefix cacheStaleness-aware truncated importance sampling