general family · filed under inference & serving
Post-training quantization
Quantizes a trained checkpoint after training, rather than learning quantization as part of training.
Also called Post-Training Quantization (PTQ).
- sources
- 3
- models
- 3
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers (Frantar et al., 2022) paper arxiv.orgseminal PTQ method for LLMs
Evidence
3 spans quoted from the sources, strongest treatment first.
Stage 5: Post-training Quantization (PTQ)
We apply post-training quantization (PTQ) using Model-Optimizer to quantize the Nemotron 3 Ultra checkpoint to NVFP4
While PTQ is already effective at preserving quality, our QAT results yield even higher overall quality compared to standard PTQ baselines.
Filed alongside
Other methods under inference & serving :: inference quantization.