Open Models Almanac
Methodsoptimizationtraining precision

single mention · optimization

Mixed-precision distributed training with BF16 parameters and FP32 reductions

Distributed training using bfloat16 parameters and float32 reductions.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training precision for the methods filed beside it.

Filed under optimization › training precision