specific method · filed under post-training
Rollout Routing Replay
Recording expert indices used during rollout and replaying those routing paths during training.
Also called Rollout Routing Replay (R3).
- sources
- 2
- models
- 3
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 3
Documented in
Evidence
3 spans quoted from the sources, strongest treatment first.
Rollout Routing Replay (R3) (Ma et al., 2025) records the expert indices used during rollout and replays them during training, reproducing the captured execution path.
usedsoftware implementationin MiMo-V2.6Xiaomi
Rollout Routing Replay (R3)
usedoptimizationin MiMo-V2-FlashXiaomi
We propose Rollout Routing Replay (R3) [ma2025stabilizing] to train RL using the same routed experts from rollout
usedtraining objectivein MiMo-V2-FlashXiaomi
Filed alongside
Other methods under post-training :: reinforcement learning algorithm.
Group Relative Policy OptimizationReinforcement LearningGroupwise Advantage RedistributionFreezing the MoE Router During Reinforcement LearningIcePopOff-Policy Sequence MaskingReinforcement Learning from Verifiable RewardsUnbiased KL EstimateAsynchronous Group Relative Policy OptimizationChain-of-Thought Reinforcement LearningDirect Double-Sided Importance SamplingKeep Sampling MaskMixed Reinforcement LearningPivot Reinforcement LearningReinforcement Learning Post-TrainingAbstention TrainingAdvantage ShapingAgentic RL Task MixCISPO with Length-Weighted Leave-One-Out Group-Relative AdvantagesConcatenated Routing ReplayDerived-Latency PenaltyDomain-Specialized RL ExpertsDomain-Specific GRPO TrainingDropping All-Zero-Advantage Groups