Open Models Almanac
Methodsoptimizationtraining precision

implementation detail · filed under optimization

NVFP4

NVIDIA's 4-bit floating-point format, used in the evidence for training or quantized weights, activations, and gradients.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See training precision for the methods filed beside it.

Also called NVFP4 4-bit Training Format, ultraefficient 4-bit NVFP4 training format, NVFP4 precision format, NVFP4 Quantization, NVFP4 checkpoint, NVFP4 quantized checkpoints.

sources
13
models
6
labs adopt it
4
strongest
core

Documented in

Evidence

16 quotes from 13 sources, one card per model, each opening with its strongest.

How sources treat it
optional 4used 8default 1core 3

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted6 sources

The majority of linear layers use NVFP4 for weights, activations, and gradients

coremodel architecturein Nemotron 3 UltraNVIDIA
8 more from this model

We apply post-training quantization (PTQ) using Model-Optimizer to quantize the Nemotron 3 Ultra checkpoint to NVFP4

usedpost trainingin Nemotron 3 UltraNVIDIA

open-sourcing the training recipes, data, and RL environments. Checkpoints ... NVFP4 quantized

usedsoftware implementationin Nemotron 3 UltraNVIDIA

use NVIDIA’s ultraefficient 4-bit NVFP4 training format on the NVIDIA Blackwell architecture

usedoptimizationin Nemotron 3 Super and UltraNVIDIA

We release a single NVFP4 checkpoint for Nemotron 3 Ultra

usedpost trainingin Nemotron 3 UltraNVIDIA

We release a single NVFP4 checkpoint for Nemotron 3 Ultra

usedinference servingin Nemotron 3 UltraNVIDIA

These figures use the final NVFP4 weights that NVIDIA recommends for inference

defaultinference servingin Nemotron 3 UltraNVIDIA

NVFP4 quantized checkpoints

usedinference servingin Nemotron 3 UltraNVIDIA

Pretrained in NVFP4.

usedotherin Nemotron 3 UltraNVIDIA

The pre-training phase used an NVFP4 recipe.

corepost trainingin Nemotron 3.5 LightningNVIDIA
1 more from this model

NVIDIA publishes both BF16 and NVFP4 results across knowledge, reasoning, coding, agents, instruction following and long context.

optionalunclearin Nemotron 3.5 LightningNVIDIA

Inkling

Thinking Machines Labadopted1 source

The NVFP4 checkpoint offers a quantized alternative that reduces the aggregated VRAM requirement to at least 600 GB.

optionalinference servingin InklingThinking Machines Lab

Step-3.7-Flash

StepFunadopted1 source

Step 3.7 Flash is also available in an NVFP4-quantized variant for efficient deployment on NVIDIA GPUs

optionalsoftware implementationin Step 3.7 Flash NVFP4StepFun

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑