The student minimizes the reverse KL𝐷KL(𝜋𝑆(·|𝑐𝑡)‖𝜋𝑇 (·|𝑐𝑡))at every position of its own responses
specific method · filed under training objective
Reverse KL divergence
Reverse KL divergence is a loss function used in the multi-teacher on-policy distillation mechanism.1 It is used to formalize the multi-teacher distillation as an on-policy reinforcement learning objective.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
The reverse KL divergence loss between student and teacher is defined as
— MiMo-V2-Flash Technical Reportwe cast multi-teacher distillation as an on-policy reinforcement learning objective
— MiMo-V2-Flash Technical Report
Also called reverse KL divergence loss, Reverse-KL on-policy distillation, The student minimizes the reverse KL.
Documented in
Evidence
2 quotes from 2 sources, one card per model, each opening with its strongest.
How sources treat it
used 1core 1
coreunclearin Nemotron 3.5 SuperNVIDIA
MiMo-V2.5
Xiaomireverse KL divergence loss between student and teacher
usedpost trainingin MiMo-V2-FlashXiaomi