Open Models Almanac
Methodsoptimizationquantization-aware training

single mention · optimization

NVFP4 Training

NVFP4 Training is quantization-aware pretraining that quantizes weight, activation, and gradient tensors to the NVFP4 number format.1 This enables the use of NVFP4 GEMMs in the forward pass and in gradient computations, instead of simulating the format around higher-precision GEMMs.2

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. Weight, activation, and gradient tensors are quantized to NVFP4 — NVIDIA Nemotron 3: Efficient and Open Intelligence
  2. which enables use of NVFP4 GEMMs in fprop, dgrad, and wgrad — NVIDIA Nemotron 3: Efficient and Open Intelligence

Also called quantized pretraining with NVFP4.

Filed under optimization › quantization-aware training