Open Models Almanac
Methodsoptimizationtraining precision

implementation detail · filed under optimization

BF16 precision

Use of the bfloat16 floating-point format for selected layers or as a general precision choice, without evidence of a particular training procedure.

The sources on this page name it without explaining how it works. The line above is the extraction's one-line gloss. See training precision for the methods filed beside it.

Also called BF16, BF16 testing, BF16 checkpoint inference.

sources
4
models
2
labs adopt it
2
strongest
used

Documented in

Evidence

4 quotes from 4 sources, one card per model, each opening with its strongest.

How sources treat it
evaluated 1optional 1used 2

NVIDIA-Nemotron-3-Ultra-550B-A55B

NVIDIAadopted3 sources

select layers (including latent projections, MTP layers, QKV/attention projections, and embeddings) are maintained in BF16 or MXFP8 for training stability

usedmodel architecturein Nemotron 3 UltraNVIDIA
2 more from this model

BF16 precision for attention linear layers, MoE latent projection layers

usedpost trainingin Nemotron 3 UltraNVIDIA

our tests show minimal intelligence impact compared to BF16 testing

evaluatedevaluation onlyin Nemotron 3 UltraNVIDIA

Further reading

Picked by hand, not extracted: where to read more, not evidence for anything on this page.

↑