single mention · optimization
Mixed-precision distributed training with BF16 parameters and FP32 reductions
Distributed training using bfloat16 parameters and float32 reductions.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training precision for the methods filed beside it.
Filed under optimization › training precision