Open Models Almanac
Methodstraining objectivedistillation objective

general family · filed under training objective

Quantization-aware distillation

A quantized student is trained to match traces from a higher-precision teacher to recover model quality.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See distillation objective for the methods filed beside it.

Also called quantization-aware distillation (QAD), QAD.

sources
4
models
3
labs adopt it
2
strongest
used

Documented in

Evidence

5 quotes from 4 sources, one card per model, each opening with its strongest.

How sources treat it
used 5

The model uses NVIDIA Model Optimizer for post-training quantization and quantization-aware distillation (QAD).

usedpost trainingin Nemotron 3.5 Super VLNVIDIA
3 more from this model

The released checkpoint uses the predefined aggressive PTQ recipe followed by SFT-based QAD at 200 iterations.

usedpost trainingin Nemotron 3.5 SuperNVIDIA

refined through quantization-aware distillation using the supervised fine-tuning data blend

usedpost trainingin Nemotron 3.5 Super VLNVIDIA

We use KL divergence as our objective.

usedpost trainingin Nemotron 3.5 SuperNVIDIA

Laguna-S-2.1

Poolsideadopted1 source

To mitigate this, we used quantization-aware distillation (QAD) to recover quality; the quantized student was trained to match the traces from the higher-precision teacher on a fixed distillation dataset.

usedpost trainingin Laguna XS.2Poolside
↑