Open Models Almanac
Methodspost-trainingpolicy distillation

specific method · filed under post-training

On-Policy Self-Distillation

A method that uses the model's own on-policy behavior with targeted hints or guidance to improve difficult tasks or failures.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See policy distillation for the methods filed beside it.

Also called OPSD, error-dependent hints, targeted OPSD.

sources
2
model
1
lab adopt it
1
strongest
used

Documented in

Aleph Alpha

Evidence

4 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 4

Kolibri-1

Aleph Alphaadopted2 sources

For hard tasks that the model can't solve, we additionally used on-policy self distillation to hint the model towards the correct solutions.

usedpost trainingin KolibriAleph Alpha
3 more from this model

OPSD aims to bootstrap the model’s capabilities via targeted hints for failures with respect to terminal code execution and tool calls.

usedtraining objectivein KolibriAleph Alpha

we implemented OPSD (Section 3.2.1.4) using error-dependent hints to provide a dense learning signal to the model.

usedtraining objectivein KolibriAleph Alpha

We additionally employ targeted OPSD (S. Zhao et al. 2026) for selected multi-turn environments

usedtraining objectivein KolibriAleph Alpha
↑