Open Models Almanac
Methodsinference & servingserving parallelism

single mention · inference & serving

FP8 parameter quantization during transfer

Quantizes transferred model parameters to FP8 on the assigned rank.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See serving parallelism for the methods filed beside it.

Also called quantised to FP8 on the assigned rank.

Filed under inference & serving › serving parallelism