MiMo-V2.6-Flash · MiMo-V2.6-Pro
XiaomiRollout Routing Replay (R3) (Ma et al., 2025) records the expert indices used during rollout and replays them during training, reproducing the captured execution path.
specific method · filed under post-training
Rollout Routing Replay (R3) is a reinforcement-learning training technique for MoE models that reuses the expert routing decisions made during rollout in the training pass.1 It addresses inconsistent expert routing between rollout and training caused by numerical precision issues.2
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Rollout Routing Replay (R3) [ma2025stabilizing] to train RL using the same routed experts from rollout— MiMo-V2-Flash Technical Report
MoE models suffer from inconsistent expert routing across rollout and training due to numerical precision issues— MiMo-V2-Flash Technical Report
Also called Rollout Routing Replay (R3).
3 quotes from 2 sources, one card per model, each opening with its strongest.
Rollout Routing Replay (R3) (Ma et al., 2025) records the expert indices used during rollout and replays them during training, reproducing the captured execution path.
We propose Rollout Routing Replay (R3) [ma2025stabilizing] to train RL using the same routed experts from rollout
Rollout Routing Replay (R3)