single mention · optimization
NVFP4 Training
NVFP4 Training is quantization-aware pretraining that quantizes weight, activation, and gradient tensors to the NVFP4 number format.1 This enables the use of NVFP4 GEMMs in the forward pass and in gradient computations, instead of simulating the format around higher-precision GEMMs.2
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
Weight, activation, and gradient tensors are quantized to NVFP4
— NVIDIA Nemotron 3: Efficient and Open Intelligencewhich enables use of NVFP4 GEMMs in fprop, dgrad, and wgrad
— NVIDIA Nemotron 3: Efficient and Open Intelligence
Also called quantized pretraining with NVFP4.
Filed under optimization › quantization-aware training