Open Models Almanac
Methodsdata curationdata mixture & curriculum

methods · data curation

data mixture & curriculum

117 methods filed at this node or below it, from the sources of 27 models.

In this branch 117

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 117

Heterogeneous training mixture core · 2 sources · 2 quotes
Heterogeneous-data co-training core · 2 sources · 4 quotes
Curated higher-quality mid-training data used · 3 sources · 4 quotes
Cross-embodiment robot training used · 2 sources · 2 quotes
KL-regularized data-mixture optimization used · 2 sources · 3 quotes
MergeMix used · 2 sources · 4 quotes
MergeMix expert-checkpoint merging used · 2 sources · 3 quotes
Vulnerability-discovery training data used · 2 sources · 2 quotes
109 from one source, or only mentioned as not used
AutoMixer core · 1 source · 2 quotes
Cross-domain and cross-trajectory shuffling core · 1 source · 2 quotes
Dataloader-time dataset mixing core · 1 source · 1 quote
Heterogeneous training data pyramid core · 1 source · 1 quote
Open X-Embodiment dataset mixture core · 1 source · 1 quote
Probabilistic multi-modal dataset mixing core · 1 source · 1 quote
Sample Mixer core · 1 source · 1 quote
Weighted dataset-proportion mixing core · 1 source · 1 quote
Weighted multi-dataset mixing core · 1 source · 1 quote
Three-bucket training curriculum default · 1 source · 1 quote
Agent-centric multimodal data mixture used · 1 source · 1 quote
Blind pairwise bucket-boundary calibration used · 1 source · 1 quote
Combined per-band sampling budgets used · 1 source · 1 quote
Cross-embodied robot data mixture used · 1 source · 1 quote
Curriculum-driven data loading used · 1 source · 1 quote
Data Scheduler used · 1 source · 1 quote
Dirichlet sampling of data-mixture weights used · 1 source · 2 quotes
Episode-level camera-order randomization used · 1 source · 3 quotes
Four-epoch dataset repetition cap used · 1 source · 1 quote
Heterogeneous data-mixture batch sampling used · 1 source · 1 quote
Interleave bulk-shuffled dataset indices used · 1 source · 1 quote
Interleaved mixed-modality training used · 1 source · 1 quote
Interleaved multimodal data used · 1 source · 1 quote
Joint multi-task training used · 1 source · 1 quote
Joint vision-language and action training used · 1 source · 1 quote
Multi-task batch mixing used · 1 source · 1 quote
Multi-task training used · 1 source · 1 quote
Multimodal web-data co-training used · 1 source · 1 quote
Octo dataset-mixture weighting used · 1 source · 1 quote
Paired side-and-wrist camera-view examples used · 1 source · 1 quote
Preserve natural training-data format used · 1 source · 1 quote
Reasoning-focused RL curriculum used · 1 source · 1 quote
Skill-specific SFT mixture design used · 1 source · 1 quote
Text-to-multimodal data annealing schedule used · 1 source · 1 quote
Three-stage data mixture strategy used · 1 source · 1 quote
Three-stage pre-training curriculum used · 1 source · 1 quote
Training on subtask-prediction data used · 1 source · 1 quote
Transparent tokenizer training mixture used · 1 source · 1 quote
Two-axis mid-training data mixing used · 1 source · 1 quote
Two-phase curriculum used · 1 source · 1 quote
Upsampling long-context data used · 1 source · 1 quote
Video-dominant pre-training data used · 1 source · 1 quote
Within-batch mixing of training sources used · 1 source · 1 quote
Weighted multi-dataset fine-tuning optional · 1 source · 1 quote
Fixed-budget Indic mixture-ratio ablations evaluated · 1 source · 1 quote
ProtocolQA-aligned RL training datasets evaluated · 1 source · 1 quote
Training data mixing evaluated · 1 source · 1 quote
Uniformly upsample German SFT data evaluated · 1 source · 1 quote
Fixed-ratio environment sampling not used · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelmethods
DeepSeek-V4.1-Flash coreFiltering, deduplication, and difficulty calibration of synthesized data core—
GLM-5.3 usedVulnerability-discovery training data used—
MiMo-V2.6-Flash coreSample Mixer coreAgent-centric multimodal data mixture usedData Scheduler usedDynamic sampling for mixed-task reinforcement learning usedSmooth weighted round-robin source scheduling used—
NVIDIA-Nemotron-3-Ultra-550B-A55B usedGaussian-based data-mixture and curriculum construction usedTwo-phase curriculum used—
FLUX 3 Action usedVideo-dominant pre-training data used—
GR00T N1.7 coreCo-training on synthetic and real-world data coreHeterogeneous training data pyramid coreHeterogeneous training mixture core1:1 co-training of real-world and neural trajectories usedHeterogeneous data-mixture batch sampling usedHeterogeneous-data co-training usedWeighted multi-dataset fine-tuning optional—
Gemma 4 31B usedIn-context attribution, hedging, and refusal data subsets used—
Kimi K3 usedUpsampling long-context data used—
Kolibri-1 coreAdaptive RL prompt and environment curriculum corePreserve pre-training dataset proportions in mid-training defaultThree-bucket training curriculum defaultBisection for shared reweighting-scale selection usedCombined per-band sampling budgets usedDirichlet sampling of data-mixture weights usedEmpirical pre- and mid-training data-mix selection usedFastText-quality gating of annotation scores usedFour-epoch dataset repetition cap usedGaussian-process surrogate optimization of annotation weights usedInterleave bulk-shuffled dataset indices usedKL-regularized data-mixture optimization usedLength-based reweighting of mid-training data usedLinear interpolation from pre-training mix to mid-training mix usedLog-linear benchmark models for optimizing mid-training domain weights usedLog-linear proxy model for predicting benchmark BPB from data mixture usedMergeMix usedMergeMix expert-checkpoint merging usedMid-rank normalization of annotation scores usedOptimize annotation mixture weights against proxy-model BPB usedPer-source repetition caps in data-mixture optimization usedProxy-model mixture selection using fitted mixing laws usedSFT sampling proportional to unique-token counts usedShard-proportional batch sampling by multivariate hypergeometric draw usedShuffle buffer for conversation-level batch balance usedTraining-data bias mitigation through reweighting and augmentation usedTwo-axis mid-training data mixing usedUpweighting selected agentic and competitive-programming environments usedWater-filling redistribution under repetition caps usedTraining data mixing evaluatedUniformly upsample German SFT data evaluatedVarying math-to-code proportions in a training data mix evaluatedFixed-ratio environment sampling not used—
LFM3-1.7B usedCurated higher-quality mid-training data usedDifficulty-ordered SFT curriculum based on ensemble solve rates usedMain pre-training with an expanded domain mix usedSkill-specific SFT mixture design usedText-to-multimodal data annealing schedule usedData annealing as an alternative to connector pre-training evaluated—
Laguna-S-2.1 coreAutoMixer coreBlind pairwise bucket-boundary calibration usedKL-regularized data-mixture optimization usedPrior-constrained Dirichlet mixture exploration usedRL task sampling proportional to unsolved rate used—
MiMo-V2.5 usedThree-stage data mixture strategy used—
MiMo-V2.5-Pro usedThree-stage data mixture strategy used—
MiMo-V2.6-Pro coreSample Mixer coreAgent-centric multimodal data mixture usedData Scheduler usedDynamic sampling for mixed-task reinforcement learning usedSmooth weighted round-robin source scheduling used—
MiniMax-M3 usedInterleaved mixed-modality training usedInterleaved multimodal data used—
Mistral-Large-4 coreHeterogeneous training mixture coreCurriculum-driven data loading used—
MolmoAct 2 coreMixed training on action-only and depth-and-action examples corePre-training on action reasoning, auxiliary robot, and multimodal web data coreWeighted sampling by assigned mixture rates coreCo-training vision-language-action models with multimodal web data usedEpisode-level camera-order randomization usedHeterogeneous embodiment-specific action representations for tokenizer training usedJoint multi-task training usedJoint refinement on embodied and general multimodal data usedMulti-task batch mixing usedPaired side-and-wrist camera-view examples usedTransparent tokenizer training mixture usedUniform sampling of action, depth, and joint depth-and-action tasks usedWithin-batch mixing of training sources used—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B usedCurated internal-data mixture for vision tool-use training usedDifficulty-based routing of validated questions to SFT or RL usedIndic continued-pretraining data allocation usedPreserve natural training-data format usedReasoning-focused RL curriculum usedStage-specific domain weighting by distillation learning curves usedFixed-budget Indic mixture-ratio ablations evaluatedMix supervised fine-tuning data with teacher-generated reinforcement-learning rollouts evaluated—
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B usedCurated internal-data mixture for vision tool-use training usedDifficulty-based routing of validated questions to SFT or RL usedIndic continued-pretraining data allocation usedPreserve natural training-data format usedReasoning-focused RL curriculum usedStage-specific domain weighting by distillation learning curves usedFixed-budget Indic mixture-ratio ablations evaluatedMix supervised fine-tuning data with teacher-generated reinforcement-learning rollouts evaluated—
OpenVLA / OpenVLA-OFT coreOpen X-Embodiment dataset mixture coreDropping a poorly fitting dataset from the training mixture usedOcto dataset-mixture weighting used—
Qwen3.5-397B-A17B usedThree-stage pre-training curriculum used—
SmolVLA usedMulti-task training used—
VLA Foundry (Foundry-VLA-1.7B) coreDataloader-time dataset mixing coreProbabilistic multi-modal dataset mixing coreWeighted dataset-proportion mixing coreWeighted multi-dataset mixing coreMulti-task training on real and simulated robotics data defaultTraining on simulated and real-world manipulation data usedExplicitly weighted batch balancing across datasets optional—
X-VLA coreCross-domain and cross-trajectory shuffling coreWeighted sampling of heterogeneous datasets core—
Xiaomi-Robotics-1 used1:6 vision-language-to-robot-trajectory sampling usedJoint training on robot trajectories and vision-language data usedJoint vision-language and action training used—
gpt-oss-120b usedText-only pre-training corpus focused on STEM, coding, and general knowledge usedProtocolQA-aligned RL training datasets evaluated—
π0.5 coreHeterogeneous-data co-training corePre-training on diverse robot-manipulation data coreCo-training on robot demonstrations and multimodal data usedCross-embodied robot data mixture usedCross-embodiment robot training usedDown-weighting overrepresented task–robot combinations usedLarge-scale multi-source robot-manipulation pre-training mixture usedMultimodal web-data co-training usedRandom camera-view selection during DROID training usedRandom selection among language annotations during DROID training usedTraining on subtask-prediction data usedUniversal action tokenizer with multiple action-space parameterizations used—
↑