specific method · filed under optimization
MXFP4 Quantization-Aware Training
MXFP4 quantization-aware training applies FP4 quantization during post-training so the model adapts to the precision degradation introduced by quantization.1 FP32 master weights are quantized to FP4 and dequantized back to FP8 for computation, with gradients propagated to the FP32 weights as through a straight-through estimator.2,3
Trade-off It targets inference acceleration and reduced memory traffic at deployment, at the cost of the precision degradation the model must adapt to.4
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
we introduce Quantization-Aware Training (QAT) [QAT] during the post-training stage, enabling the model, including those of teacher and reference models, to adapt to the precision degradation introduced by quantization
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencethe FP32 master weights maintained by the optimizer are first quantized to FP4, then dequantized back to FP8 for computation
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencegradients are computed with respect to the same FP8 weights in the forward pass and directly propagated back to the FP32 master weights, equivalent to applying the Straight-Through Estimator (STE
— DeepSeek-V4: Towards Highly Efficient Million-Token Context IntelligenceTo achieve inference acceleration and reducing memory traffic at deployment
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Also called FP4 Quantization-Aware Training, MXFP4 quantization-aware training (QAT), MXFP4 quantization-aware post-training.