single mention · training objective
Teacher-weighted reverse-KL surrogate for Top-K distillation
Teacher-weighted surrogate is a variant of the reverse KL used in distillation objectives.1
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Teacher-weighted surrogate
— LFM2 Technical Report
Also called Teacher-weighted surrogate.
Filed under training objective › distillation objective