single mention · inference & serving
FP8 parameter quantization during transfer
Quantizes transferred model parameters to FP8 on the assigned rank.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See serving parallelism for the methods filed beside it.
Also called quantised to FP8 on the assigned rank.
Filed under inference & serving › serving parallelism