The model uses NVIDIA Model Optimizer for post-training quantization and quantization-aware distillation (QAD).
3 more from this model
The released checkpoint uses the predefined aggressive PTQ recipe followed by SFT-based QAD at 200 iterations.
refined through quantization-aware distillation using the supervised fine-tuning data blend
We use KL divergence as our objective.