Open Models Almanac
Methodsinference & servinginference quantization

single mention · inference & serving

W4A4 quantization

W4A4 quantizes both the weights and the activations of a layer to four bits. On hardware whose matrix units multiply four-bit numbers, such as NVIDIA Blackwell, it has the highest peak arithmetic of the 4-bit layouts, and it puts the activations on the same coarse grid as the weights.

Written for the explainer Quantization for serving: which bits to remove, and what each removal saves.

Also called W4A4.

Filed under inference & serving › inference quantization