Open Models Almanac
Methodspost-trainingpolicy distillation

filed under post-training

Multi-Teacher On-Policy Distillation

Multi-Objective Policy Distillation distills several specialized teachers into a single student model, each teacher supervising the domain it owns.1,2 It is on-policy: the student generates responses to prompts, and at every token its reverse KL divergence to the domain teacher is minimized.3,4

Trade-off MOPD recovers a large fraction of the gap between the RL-trained student and the specialized teachers across domains.5

How this definition was written

Summarised by glm-5.3-flash from 2 sources; every sentence is checked against the quote it cites.

  1. multi-teacher on-policy distillation (MOPD) stage after RL which distills the capabilities of the specialized teachers of the preceding subsections into a single student model — NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  2. the teacher that owns the domain is consulted — NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  3. Given a prompt from some domain, the student generates a response — NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  4. The student minimizes the reverse KL — NVIDIA Nemotron 3.5 Super and Lightning Technical Report
  5. MOPD results across domains, showing gains over the RLVR student and recovery toward specialized teachers — NVIDIA Nemotron 3 Ultra Technical Report

Also called MOPD, Multi-Domain On-Policy Distillation, Mixture of On-Policy Distillation, Multi-Domain On-Policy Distillation (MOPD), Multi-Teacher On-Policy Distillation, multi-teacher On-Policy Distillation (OPD).

sources
21
models
10
labs adopt it
6
strongest
core

Documented in

Evidence

29 quotes from 21 sources, one card per model, each opening with its strongest.

How sources treat it
used 21core 8

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted5 sources

we augment the pipeline with Multi-teacher On-Policy Distillation (MOPD) (Yang et al., 2026; Lu & Lab, 2025; Xiao et al., 2026), enabling both broad capability acquisition and targeted specialization.

corepost trainingin Nemotron 3 UltraNVIDIA
8 more from this model

MOPD provides a dense token-level learning signal from the relevant teacher distribution.

coreunclearin Nemotron 3 UltraNVIDIA

Post-trained with enhanced pipeline involving Supervised Fine Tuning (SFT), Reinforcement Learning (RL), and Multi-teacher On-Policy Distillation (MOPD) for improved model accuracy.

usedpost trainingin Nemotron 3 UltraNVIDIA

The model underwent Multi-Domain On-Policy Distillation (MOPD) to improve reasoning across many task types while staying efficient

usedoptimizationin Nemotron 3 UltraNVIDIA

Multi-Domain On-Policy Distillation (MOPD) to improve reasoning across many task types while staying efficient

usedtraining objectivein Nemotron 3 UltraNVIDIA

MOPD trains the student to match the corresponding teacher on states induced by the student itself

usedoptimizationin Nemotron 3 UltraNVIDIA

post-trained using SFT, RL, and MOPD

usedpost trainingin Nemotron 3 UltraNVIDIA

Multi-teacher On-Policy Distillation (MOPD) consolidated these teachers into Ultra

corepost trainingin Nemotron 3 UltraNVIDIA

Multi-teacher On-Policy Distillation (MOPD) consolidated these teachers into Ultra

usedpost trainingin Nemotron 3 UltraNVIDIA

MiMo-V2.5

Xiaomiadopted4 sources

through its hybrid Sliding Window Attention architecture, lightweight Multi-Token Prediction, and the MOPD post-training paradigm.

corepost trainingin MiMo-V2-FlashXiaomi
4 more from this model

MiMo-V2-Flash introduces a novel Multi-Teacher On-Policy Distillation (MOPD) paradigm

corepost trainingin MiMo-V2-FlashXiaomi

Post-training incorporates SFT, large-scale agentic RL, and Multi-Teacher On-Policy Distillation (MOPD)

usedpost trainingin MiMo-V2.5Xiaomi

Post-training incorporates SFT, large-scale agentic RL, and Multi-Teacher On-Policy Distillation (MOPD)

usedpost trainingin MiMo-V2.5Xiaomi

RL and MOPD

usedoptimizationin MiMo-V2.5Xiaomi

Kimi K3

Moonshot AIadopted1 source

We adopt Multi-Teacher On-Policy Distillation (MOPD) to consolidate these domain-specialized capabilities across varying reasoning efforts into a unified model

coreunclearin Kimi K3Moonshot AI
1 more from this model

Domain- and effort-specialized policies are consolidated into a unified model through multi-teacher on-policy distillation [75, 134, 29].

usedpost trainingin Kimi K3Moonshot AI

During post-training, multi-teacher on-policy distillation (MOPD) combines guidance from specialized teachers across text and vision.

usedpost trainingin Nemotron 3.5 Super VLNVIDIA
6 more from this model

After MOPD, truncation decreases to 0.3% with 13,669 output tokens per rollout, where only one runaway loop failure is captured.

usedpost trainingin Nemotron 3.5 Super and LightningNVIDIA

Multi-teacher on-policy distillation consolidates these capabilities into a single model.

usedpost trainingin Nemotron 3.5 Super VLNVIDIA

Multi-teacher on-policy distillation consolidates these capabilities into a single model.

usedpost trainingin Nemotron 3.5 Super VLNVIDIA

Nemotron 3.5 Super included a multi-teacher on-policy distillation (MOPD) stage after RL

usedunclearin Nemotron 3.5 SuperNVIDIA

specialized teacher training, and Multi-Teacher On-Policy Distillation (MOPD)

usedpost trainingin Nemotron 3.5 SuperNVIDIA

before consolidating their capabilities through MOPD

usedpost trainingin Nemotron 3.5 SuperNVIDIA

LFM3-1.7B

Liquid AIadopted3 sources

six specialist RL teachers (code, math, tool use, long context, agentic work, and enterprise workflows) are distilled back into a single model.

usedpost trainingin LFM3-1.7BLiquid AI
2 more from this model

six specialist RL teachers (code, math, tool use, long context, agentic work, and enterprise workflows) are distilled back into a single model

usedpost trainingin LFM3-1.7BLiquid AI

Post-training from this checkpoint (SFT, mixed RL, multi-domain on-policy distillation, and agentic RL) produces LFM3-1.7B.

usedpost trainingin LFM3-1.7BLiquid AI

Mistral-Large-4

Mistral AIadopted1 source

We combine these teachers into a single model through multi-teacher on-policy distillation [45, 46].

usedunclearin Mistral Large 4Mistral AI

After MOPD, truncation decreases to 0.3% with 13,669 output tokens per rollout, where only one runaway loop failure is captured.

usedpost trainingin Nemotron 3.5 Super and LightningNVIDIA
2 more from this model

Nemotron 3.5 Super included a multi-teacher on-policy distillation (MOPD) stage after RL

usedunclearin Nemotron 3.5 SuperNVIDIA

before consolidating their capabilities through MOPD

usedpost trainingin Nemotron 3.5 SuperNVIDIA
↑