Open Models Almanac
Methodsoptimizationquantization-aware training

single mention · optimization

FP4 Quantization

FP4 quantization-aware training trains a model to adapt to the precision loss of FP4 quantization, applied here to MoE expert weights and the CSA indexer's Query-Key path.1,2 It targets inference acceleration and lower memory traffic at deployment, with FP4 weights quantized from FP32 masters and dequantized back to FP8 for computation.3,4

Trade-off It speeds up attention score computation and the top-k selector while preserving KV-entry recall.5,6

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we introduce Quantization-Aware Training (QAT) [QAT] during the post-training stage, enabling the model, including those of teacher and reference models, to adapt to the precision degradation introduced by quantization — DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
  2. We apply FP4 (MXFP4) quantization [OCP_MXFormat] to two components: (1) MoE expert weights, which are a major source of GPU memory occupancy [gpt_oss], and (2) the Query-Key (QK) path in the indexer of CSA — DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
  3. To achieve inference acceleration and reducing memory traffic at deployment — DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
  4. the FP32 master weights maintained by the optimizer are first quantized to FP4, then dequantized back to FP8 for computation — DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
  5. accelerating attention score computation in long-context scenarios — DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
  6. This optimization achieves a 2 speedup for the top-k selector, while preserving a 99.7% recall rate of KV entries — DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence

Also called FP4 (MXFP4) quantization.

Filed under optimization › quantization-aware training

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.