Model techniques map
Taxonomyoptimizationtraining precision

taxonomy node · level 2

training precision

19 methods filed at this node or below it, from the sources of 12 models.

optimization :: training precision

Matching aids for the classifier: FP8 mixed precision training; NVFP4 pre-training; MXFP8; BF16; FP32 gradient reduction; FP4+FP8 mixed precision.

In this branch 19

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 19

NVFP4 core · 10 sources · 13 quotes
NVFP4 pre-training core · 5 sources · 5 quotes
BF16 mixed-precision training default · 1 source · 1 quote
BF16 used · 5 sources · 5 quotes
FP8 mixed-precision training used · 4 sources · 4 quotes
FP4+FP8 mixed precision used · 2 sources · 2 quotes
FP8-precision reinforcement learning used · 2 sources · 2 quotes
MXFP8 used · 2 sources · 2 quotes
Block-wise FP8 activation quantization with offload used · 1 source · 1 quote
E2M1 used · 1 source · 1 quote
FP32 attention-output retention used · 1 source · 1 quote
FP32 gradient reduction used · 1 source · 1 quote
FP8 storage for the residual state used · 1 source · 1 quote
High-precision final network layers used · 1 source · 1 quote
Mixed-FP8 quantization used · 1 source · 1 quote
Mixed-precision training used · 1 source · 1 quote
NVFP4 fine-grained micro-block scaling used · 1 source · 1 quote
Two-dimensional block quantization used · 1 source · 1 quote
BF16 gradient reduction not used · 1 source · 1 quote

By model

Which of this branch's techniques each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modeltechniques
NVIDIA-Nemotron-3-Ultra-550B-A55B coreNVFP4 coreNVFP4 pre-training coreBF16 usedE2M1 usedFP32 gradient reduction usedMixed-FP8 quantization usedMXFP8 usedTwo-dimensional block quantization usedBF16 gradient reduction not used—
DeepSeek-V4-Flash usedFP4+FP8 mixed precision used—
MiMo-V2.5 usedFP8 mixed-precision training used—
GLM-5.2 usedMixed-precision training used—
DeepSeek-V4-Pro usedFP4+FP8 mixed precision used—
Inkling optionalNVFP4 optional—
Kimi K3 usedBlock-wise FP8 activation quantization with offload usedFP32 attention-output retention used—
Laguna-S-2.1 defaultBF16 mixed-precision training defaultFP8-precision reinforcement learning used—
MiMo-V2.5-Pro usedFP8 mixed-precision training used—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B coreNVFP4 coreBF16 usedHigh-precision final network layers usedMXFP8 usedNVFP4 fine-grained micro-block scaling usedNVFP4 pre-training used—
Qwen3.8-Flash-Next usedFP8 storage for the residual state used—
Step-3.7-Flash usedBF16 usedNVFP4 optional—