Open Models Almanac
Methodsoptimizationtraining precision

methods · optimization

training precision

30 methods filed at this node or below it, from the sources of 21 models.

In this branch 30

Everything filed at this node or below it, with one collapsible heading per child node.

filed here 30

NVFP4 core · 13 sources · 16 quotes
NVFP4 pretraining core · 5 sources · 5 quotes
BF16 mixed-precision training default · 5 sources · 5 quotes
BF16 precision used · 4 sources · 4 quotes
FP8 mixed-precision training used · 4 sources · 4 quotes
FP4 and FP8 mixed precision used · 2 sources · 2 quotes
FP8-precision reinforcement learning used · 2 sources · 2 quotes
Mixed-precision training used · 2 sources · 3 quotes
22 from one source, or only mentioned as not used
BF16 attention projections used · 1 source · 1 quote
Blockwise scales in generator and trainer used · 1 source · 1 quote
E2M1 floating-point format used · 1 source · 1 quote
FP32 attention-output retention used · 1 source · 1 quote
FP32 gradient reduction used · 1 source · 1 quote
FP32 language-model head used · 1 source · 1 quote
FP8 quantization of selected layers used · 1 source · 1 quote
FP8 residual-state storage used · 1 source · 1 quote
FP8 tensor-core training used · 1 source · 1 quote
High-precision final network layers used · 1 source · 1 quote
MXFP8 used · 1 source · 1 quote
MXFP8 Mamba output projections used · 1 source · 1 quote
NVFP4 fine-grained micro-block scaling used · 1 source · 1 quote
Two-dimensional block quantization used · 1 source · 1 quote
Automatic mixed precision optional · 1 source · 1 quote
BF16 expert activations evaluated · 1 source · 1 quote
BF16 language-model head evaluated · 1 source · 1 quote
BF16 gradient reduction not used · 1 source · 1 quote

By model

Which of this branch's methods each model's own documents describe, and how strongly. Under each model: its strongest treatment anywhere in the branch.

Modelmethods
Step-5-Preview optionalBF16 precision optional—
NVIDIA-Nemotron-3-Ultra-550B-A55B coreNVFP4 coreNVFP4 pretraining coreBF16 precision usedE2M1 floating-point format usedFP32 gradient reduction usedFP8 quantization of selected layers usedMXFP8 usedTwo-dimensional block quantization usedBF16 gradient reduction not used—
DeepSeek-V4-Flash usedFP4 and FP8 mixed precision used—
DeepSeek-V4-Pro usedFP4 and FP8 mixed precision used—
GLM-5.2 usedMixed-precision training used—
Inkling optionalNVFP4 optional—
Kimi K3 usedBlock-wise FP8 activation quantization with offload usedFP32 attention-output retention used—
Kolibri-1 usedMixed-precision distributed training with BF16 parameters and FP32 reductions used—
Laguna-S-2.1 defaultBF16 mixed-precision training defaultFP8-precision reinforcement learning used—
LingBot-VLA 2.0 usedBF16 mixed-precision training usedMixed-precision policy with BF16 storage and communication and FP32 reductions used—
MiMo-V2.5 usedFP8 mixed-precision training used—
MiMo-V2.5-Pro usedFP8 mixed-precision training used—
Mistral-Large-4 usedBlockwise scales in generator and trainer usedFP8 tensor-core training usedNVFP4 optional—
MolmoAct 2 usedBF16 mixed-precision training usedFP32 layer normalization and rotary position embeddings used—
NVIDIA-Nemotron-3.5-Lightning-30B-A3B coreNVFP4 coreBF16 attention projections usedFP32 language-model head usedHigh-precision final network layers usedMXFP8 Mamba output projections usedNVFP4 fine-grained micro-block scaling usedNVFP4 pretraining usedBF16 expert activations evaluatedBF16 language-model head evaluated—
NVIDIA-Nemotron-3.5-Super-VL-120B-A12B coreNVFP4 coreFP32 language-model head usedBF16 expert activations evaluatedBF16 language-model head evaluated—
OpenVLA / OpenVLA-OFT optionalAutomatic mixed precision optional—
Qwen3.8-Flash-Next usedFP8 residual-state storage used—
SmolVLA usedBF16 mixed-precision training used—
Step-3.7-Flash optionalNVFP4 optional—
VLA Foundry (Foundry-VLA-1.7B) usedMixed-precision training used—
↑