Open Models Almanac
Methodsdata curationdata mixture & curriculum

single mention · data curation

Weighted multi-dataset mixing

Weighted multi-dataset mixing trains on several datasets together, controlling how much of each batch comes from each dataset.1 A weighting parameter sets the batch balancing ratios, so a 1:2:1 weighting draws 25%, 50% and 25% of each batch from the respective datasets.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. supporting multiple datasets is as simple as adding elements to the list — VLA Foundry: A Unified Framework for Training Vision-Language-Action Models
  2. the --data.dataset_weighting parameter, which handles the batch balancing ratios; a 1:2:1 weighting corresponds to 25%/50%/25% of each batch drawn from the respective datasets — VLA Foundry: A Unified Framework for Training Vision-Language-Action Models

Also called dataset mixing.

Filed under data curation › data mixture & curriculum