Open Models Almanac
Methodsoptimizationtraining precision

single mention · optimization

FP32 layer normalization and rotary position embeddings

Computing layer normalization and rotary position embeddings in FP32 as exceptions to the surrounding precision policy.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training precision for the methods filed beside it.

Filed under optimization › training precision