Open Models Almanac
Methodstraining objectivedistillation objective

specific method · filed under training objective

Logit matching

The student is trained to match the teacher’s predictive distribution rather than only sampled-token targets.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See distillation objective for the methods filed beside it.

sources
2
model
1
labs adopt it
0
strongest
evaluated

Documented in

Evidence

3 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
evaluated 1not used 2

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAdiscussed2 sources

A natural alternative to the sampled-token objective is distribution-level distillation, where the student is trained to match the teacher’s predictive distribution

evaluatedoptimizationin Nemotron 3 UltraNVIDIA
2 more from this model

In our preliminary experiments, these objectives did not improve MOPD performance and consistently underperformed the sampled-token objective

not usedtraining objectivein Nemotron 3 UltraNVIDIA

In our preliminary experiments, these objectives did not improve MOPD performance

not usedunclearin Nemotron 3 UltraNVIDIA
↑