single mention · training objective
Cross-encoder-to-ColBERT knowledge distillation
Cross-encoder-to-ColBERT knowledge distillation trains a ColBERT student model to match relevance scores computed by a cross-encoder teacher.1 It minimizes the mean squared error between student and teacher relevance scores, giving richer continuous training signals than binary contrastive labels.2
Trade-off Distillation yields better ColBERT models than pure contrastive learning by providing richer training signals through continuous relevance scores.3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
leveraging pre-computed relevance scores from a powerful cross-encoder teacher model
— LFM2 Technical Reportminimizes the mean squared error between student and teacher relevance scores
— LFM2 Technical Reportricher training signals through continuous relevance scores rather than binary positive/negative labels
— LFM2 Technical Report
Also called Knowledge distillation framework.
Filed under training objective › distillation objective