general family · filed under data curation
Heterogeneous-data co-training
Heterogeneous-data co-training structures training corpora as a data pyramid, organizing heterogeneous sources by scale from web data to real-world robot execution.1 This strategy learns across all data layers by inferring pseudo-actions for action-less videos using a latent-action codebook and a trained inverse dynamics model.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
we structure the VLA training corpora as a data pyramid, illustrated in Fig. 1. Rather than treating the training datasets as a homogeneous pool, we organize heterogeneous sources by scale: large quantities of web data and human videos lay the base of the pyramid; synthetic data generated with physics simulations and/or augmented by off-the-shelf neural models form the middle layer, and real-world data collected on the physical robot hardware complete the top
— GR00T N1: An Open Foundation Model for Generalist Humanoid RobotsWe develop an effective co-training strategy to learn across the entire data pyramid in both pre- and post-training phases. To train our model with action-less data sources, such as human videos and neural-generated videos, we learn a latent-action codebook (Ye et al., 2025) and also use a trained inverse dynamics model (IDM) to infer pseudo-actions
— GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
Also called co-training strategy, Co-training on diverse data sources, diverse data co-training, co-training recipe, co-training framework.