specific method · filed under optimization
FP4 Quantization-Aware Training
Quantization-aware training using FP4, specifically MXFP4 in the evidence, to adapt model components to low-precision computation.
Also called MXFP4 quantization-aware training, MXFP4 quantization-aware training (QAT).
- sources
- 3
- models
- 4
- labs adopt it
- 2
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
3 spans quoted from the sources, strongest treatment first.
We employ MXFP4 quantization-aware training (QAT) during mid-training, allowing the model to adapt to low-precision computation while preserving model quality.
we incorporate FP4 quantization-aware training for MoE expert weights and the indexer QK path
FP4 Quantization-Aware Training applies MXFP4 quantization to MoE expert weights and the indexer QK path during post-training.
Filed alongside
Other methods under optimization :: quantization-aware training.