Open Models Almanac
Methodsinference & servinginference quantization

single mention · inference & serving

MoE-layer quantization

A broad approach quantizing MoE layers in formats such as FP8, INT4, or NVFP4, and potentially the KV cache, without one specific algorithm.

The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See inference quantization for the methods filed beside it.

Also called quantization of MoE layers.

Filed under inference & serving › inference quantization