single mention · optimization
FP4 Quantization
FP4 quantization-aware training trains a model to adapt to the precision loss of FP4 quantization, applied here to MoE expert weights and the CSA indexer's Query-Key path.1,2 It targets inference acceleration and lower memory traffic at deployment, with FP4 weights quantized from FP32 masters and dequantized back to FP8 for computation.3,4
Trade-off It speeds up attention score computation and the top-k selector while preserving KV-entry recall.5,6
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
we introduce Quantization-Aware Training (QAT) [QAT] during the post-training stage, enabling the model, including those of teacher and reference models, to adapt to the precision degradation introduced by quantization
— DeepSeek-V4: Towards Highly Efficient Million-Token Context IntelligenceWe apply FP4 (MXFP4) quantization [OCP_MXFormat] to two components: (1) MoE expert weights, which are a major source of GPU memory occupancy [gpt_oss], and (2) the Query-Key (QK) path in the indexer of CSA
— DeepSeek-V4: Towards Highly Efficient Million-Token Context IntelligenceTo achieve inference acceleration and reducing memory traffic at deployment
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencethe FP32 master weights maintained by the optimizer are first quantized to FP4, then dequantized back to FP8 for computation
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligenceaccelerating attention score computation in long-context scenarios
— DeepSeek-V4: Towards Highly Efficient Million-Token Context IntelligenceThis optimization achieves a 2 speedup for the top-k selector, while preserving a 99.7% recall rate of KV entries
— DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Also called FP4 (MXFP4) quantization.
Filed under optimization › quantization-aware training
Further reading
Picked by hand, not extracted: where to read more, not evidence for anything on this page.
- Microscaling Data Formats for Deep Learning (Rouhani et al., 2023) paper arxiv.org
- OCP Microscaling Formats (MX) Specification v1.0 docs www.opencompute.org