Open Models Almanac
Methodsdata curationdata mixture & curriculum

single mention · data curation

Sample Mixer

Sample Mixer is a scheduling method for asynchronous mixed-task RL that keeps training on a specified distribution across data sources.1,2 It fills the target distribution with four mechanisms: adaptive rollout concurrency and scheduling, predictive rollout dispatch, and sample replay for startup and recovery.3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. wehavemaintaineda Data Schedulerthattargets a specified training distribution across data sources — MiMo V2 6 technical report
  2. Mixed-task RL must preserve this distribution de- spite large variations in rollout duration and filtering rates — MiMo V2 6 technical report
  3. we implement theSample Mixer, with four mech- anisms filling the specified training distribution:Adaptive Rollout Concurrency sets per-source budgets, Adaptive Rollout Scheduling selects data sources within the budgets,Predictive Rollout Dispatchplaces new rollouts across ranks, andSample Replay covers startup and recovery — MiMo V2 6 technical report

Filed under data curation › data mixture & curriculum