Techniquesoptimizationquantization-aware training

implementation detail · filed under optimization

Straight-Through Estimator

An estimator used to pass gradients through the quantization operation during QAT.

source: 1
models: 2
lab adopt it: 1
strongest: used

How sources treat it

One count per evidence span, weakest treatment to strongest.

used 1

Documented in

DeepSeek-V4-Flash DeepSeek-V4-Pro

Evidence

1 span quoted from the sources, strongest treatment first.

equivalent to applying the Straight-Through Estimator (STE) through the quantization operation

usedoptimizationin DeepSeek-V4DeepSeek

DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence·DeepSeek·2026-04-26

Filed alongside

Other methods under optimization :: quantization-aware training.

Quantization-Aware Training FP4 Quantization FP4 Quantization-Aware Training MXFP4 Weights with MXFP8 Activations Stochastic Rounding INT4 Quantization-Aware Training MXFP4 Quantization-Aware Post-Training NVFP4 Training Per-Block Scalar Scaling Q4_0 Quantization Format Quantize-Dequantize Training Random Hadamard Transforms Stochastic Rounding for Mamba Cache Stochastic Rounding of Gradients