Open Models Almanac
Methodspost-trainingpolicy distillation

single mention · post-training

Model Distillation

Model distillation transfers a larger teacher model's capabilities into a smaller student model.1 It works by supervised fine-tuning the student on data generated by the teacher, spanning a range of tasks.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. transferring MiMo’s agentic experience into a smaller model — MiMo V2 6 technical report
  2. supervised fine-tuning Qwen3.5-9B (Qwen Team, 2025) on MiMo-generated data spanning coding, general-domain, visual, and cybersecurity tasks — MiMo V2 6 technical report

Also called distillation.

Filed under post-training › policy distillation