single mention · post-training
Specialize-then-rehearse training
Specialize-then-rehearse training is a two-stage fine-tuning recipe that adapts an existing released checkpoint to a new domain instead of re-running full training from scratch.1 Stage 1 fine-tunes heavily on the new-domain corpus (plus some text data to preserve language ability), and Stage 2 continues training on a mixture that interleaves the new data with the original mid-training data to balance the two.2,3
Trade-off It targets a Pareto trade-off between new-domain benchmarks and the base model's general benchmarks, with the mixture ratio swept to find the best balance.4
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
To avoid the redundant compute of re-running Molmo2’s full multimodal training with our new corpus integrated, we build on the released Molmo2 checkpoint with a two-stage specialize-then-rehearse recipe
— MolmoAct2 Action Reasoning Models for Real-World Deploymentwe fine-tune for K steps on the Molmo2-ER corpus augmented with Tulu-3 (Lambert et al., 2025) text-only data to preserve language competence
— MolmoAct2 Action Reasoning Models for Real-World Deploymenta mixture that interleaves our embodied corpus with Molmo2’s original multimodal mid-training data
— MolmoAct2 Action Reasoning Models for Real-World DeploymentSweeping , we find yields the best Pareto trade-off between the embodied-reasoning benchmarks in Table 3 and Molmo2’s general benchmarks
— MolmoAct2 Action Reasoning Models for Real-World Deployment
Also called specialize-then-rehearse recipe.
Filed under post-training