Open Models Almanac
Methodsoptimizationtraining precision

implementation detail · filed under optimization

FP4 and FP8 mixed precision

A mixed-precision scheme using FP4 for MoE expert parameters and FP8 for most other parameters.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See training precision for the methods filed beside it.

Also called FP4 + FP8 Mixed.

sources
2
models
2
lab adopt it
1
strongest
used

Documented in

Evidence

2 quotes from 2 sources, one card per model, each opening with its strongest.

How sources treat it
used 2
↑