Open Models Almanac
Methodsdata curationdata mixture & curriculum

single source · data curation

AutoMixer

AutoMixer is a framework for automated optimization of pre-training data mixtures.1 It trains a swarm of small proxy models on different data compositions, fits surrogate regressors mapping mixture proportions to downstream metrics, and optimizes the mixture over the learned surrogate.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we developedAutoMixer, a framework for automated data mixture optimization — laguna m1 xs2 technical report
  2. training a swarm of proxy models, each trained on a different data composition — laguna m1 xs2 technical report

Also called Automated data-mixture optimization.

Filed under data curation › data mixture & curriculum