specific method · filed under optimization
FP8 storage for the residual state
The widened residual state is stored in FP8 to reduce bytes moved relative to BF16.
Also called FP8 storage for the widened residual state, keep the residual state in FP8.
- source
- 1
- model
- 1
- lab adopt it
- 1
- strongest
- used
How sources treat it
One count per evidence span, weakest treatment to strongest.
used 1
Documented in
Evidence
1 span quoted from the sources, strongest treatment first.
Storing the branches in FP8 halves the bytes moved for the residual state relative to BF16, with almost no loss in quality.
usedinference servingin Qwen3.8-NextQwen
Filed alongside
Other methods under optimization :: training precision.
NVFP4BF16NVFP4 pre-trainingFP8 mixed-precision trainingFP4+FP8 mixed precisionFP8-precision reinforcement learningMXFP8BF16 gradient reductionBF16 mixed-precision trainingBlock-wise FP8 activation quantization with offloadE2M1FP32 attention-output retentionFP32 gradient reductionHigh-precision final network layersMixed-FP8 quantizationMixed-precision trainingNVFP4 fine-grained micro-block scalingTwo-dimensional block quantization