Open Models Almanac
Methodsoptimizationtraining precision

specific method · filed under optimization

BF16 mixed-precision training

Mixed-precision training using bfloat16 for most operations, with some evidence specifying FP32 master weights or retaining sensitive operations in full precision.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training precision for the methods filed beside it.

Also called mixed-precision training using BF16, bfloat16 mixed-precision training, bfloat16 precision, Mixed-precision training with bfloat16.

sources
5
models
4
labs adopt it
4
strongest
default

Documented in

Poolside
Allen Institute for AI
Hugging Face
Robbyant (Ant Group)

Evidence

5 quotes from 5 sources, one card per model, each opening with its strongest.

How sources treat it
used 4default 1

MolmoAct 2

Allen Institute for AIadopted2 sources

We use automatic mixed precision with bfloat16 for most operations, while keeping numerically sensitive operations such as layer normalization and RoPE in full precision.

usedunclearin MolmoAct2 training implementationAllen Institute for Artificial Intelligence (AI2)
1 more from this model

we enable Automatic Mixed Precision (AMP) with bfloat16 for most operations.

usedoptimizationin MolmoActAllen Institute for Artificial Intelligence (AI2)
↑