general family · filed under post-training
On-Policy Distillation
On-Policy Distillation is a post-training method that transfers the capabilities of teacher (expert) models into a single student model.1 The student learns from the teachers' output distributions on trajectories the student itself generates, optimizing a reverse KL loss so each relevant expert guides the current task context.2,3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
an effective post-training paradigm for efficiently transferring the knowledge and capabilities of domain experts to a single, unified model
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencehaving the student learn from the output distributions of teacher models on its own generated trajectories
— DeepSeek-V4: Towards Highly Efficient Million-Token Context IntelligenceComputing the reverse KL loss requires sampling training trajectories from the student to maintain on-policy learning
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Also called on-policy distillation (OPD).