single mention · data curation
Two-phase curriculum
A curriculum transitioning from a mixture biased toward dataset diversity to one biased toward data quality.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See data mixture & curriculum for the methods filed beside it.
Filed under data curation › data mixture & curriculum
Explainer · 01 · 35 minThe anatomy of a training recipeLabs agree on the rough order of stages and a few habits inside them. Each stage spends tokens on what the last one could not teach, and each can lose some of what came beforeExplainer · D2 · 35 minData mixture and curriculum: choosing what to read, and whenFitting a data mix to an objective makes gains according to that objective, but risks losing what you don't measure, and is still affected by your chosen prior