general family · filed under inference & serving
NVFP4 quantization
Uses NVIDIA’s 4-bit floating-point format for model weights; the evidence includes checkpoints and mixed-precision deployments but does not define one common recipe.
Also called NVFP4 4-bit floating-point quantization, NVFP4 checkpoint, NVFP4 mixed-precision quantization, NVFP4, NVFP4 (NVIDIA 4-bit floating point), NVFP4 quantization checkpoint.
- sources
- 5
- models
- 5
- labs adopt it
- 4
- strongest
- default
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Pushing Intelligence to 4-bit explainer research.nvidia.comNVIDIA's own writeup of the NVFP4 format
Evidence
5 spans quoted from the sources, strongest treatment first.
Only the routed experts are quantized to NVFP4; shared experts and the qkvr linears remain BF16.
Released under Apache 2.0 with BF16, FP8, NVFP4, and GGUF weights on Hugging Face
We recommend using the NVFP4 checkpoint nvidia/Qwen3.5-397B-A17B-NVFP4 for optimal serving efficiency on GB200 nodes.
Nemotron 3.5 Lightning ships with an NVFP4 checkpoint alongside BF16, using the same specialized NVFP4 kernels that power Nemotron 3 Ultra across NVIDIA Blackwell, NVIDIA Hopper, and NVIDIA Ampere GPUs.
NVIDIA's ModelOpt NVFP4 checkpoint
Filed alongside
Other methods under inference & serving :: inference quantization.