select layers (including latent projections, MTP layers, QKV/attention projections, and embeddings) are maintained in BF16 or MXFP8 for training stability
2 more from this model
BF16 precision for attention linear layers, MoE latent projection layers
our tests show minimal intelligence impact compared to BF16 testing