Open Models Almanac
Methodstraining objectivedistillation objective

single mention · training objective

Mean squared error distillation of relevance scores

Student relevance scores are trained to minimize mean squared error against teacher relevance scores.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See distillation objective for the methods filed beside it.

Also called distillation loss.

Filed under training objective › distillation objective