Open Models Almanac
Methodsinference & servinginference quantization

single mention · inference & serving

W4A8 quantization

A mixed-precision quantization strategy that applies different bit precisions to different model components, compressing MoE experts to W4A8 (INT4) while other blocks use W8A8.1 It is used to drastically reduce memory footprint without significant accuracy loss, with outlier-suppression and scaling-calibration algorithms maintaining stability in low-bit deployment.2,3

How this definition was written

Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.

  1. we applied specific precisions to different model components: standard Attention and MLP blocks use W8A8 (INT8), while the MoE experts are compressed to W4A8 (INT4 — GLM-5: from Vibe Coding to Agentic Engineering
  2. to drastically reduce memory footprint without significant accuracy loss — GLM-5: from Vibe Coding to Agentic Engineering
  3. QuaRot [2] for outlier suppression and Flex_AWQ_SSZ for scaling calibration were employed to maintain stability in low-bit deployment — GLM-5: from Vibe Coding to Agentic Engineering

Also called W4A8 mixed-precision quantization strategy.

Filed under inference & serving › inference quantization