specific method · filed under post-training
Multi-Prefix Multi-Teacher On-Policy Distillation
MOPD2 is an on-policy distillation method that combines capabilities from domain-specific teachers, including tasks that are hard to verify.1 A source trajectory with k assistant turns yields k complete history prefixes, each initializing a separate student-generated turn for token-level distillation by the relevant domain teacher.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
use Multi-Prefix Multi-Teacher On-Policy Distillation (MOPD2) to combine capabilities from teachers trained for different tasks, including tasks that are hard to verify
— MiMo V2 6 technical reportA source trajectorywith 𝑘assistant-turndecisionpointsyields 𝑘completehistoryprefixes ℎ𝑖,eachofwhich can initialize a separate student-generated turn 𝑦𝑖 for token-level distillation by the relevant domain teacher
— MiMo V2 6 technical report