single mention · inference & serving
W4A8 quantization
A mixed-precision quantization strategy that applies different bit precisions to different model components, compressing MoE experts to W4A8 (INT4) while other blocks use W8A8.1 It is used to drastically reduce memory footprint without significant accuracy loss, with outlier-suppression and scaling-calibration algorithms maintaining stability in low-bit deployment.2,3
How this definition was written
Summarised by glm-5.3-flash from 1 source; every sentence is checked against the quote it cites.
we applied specific precisions to different model components: standard Attention and MLP blocks use W8A8 (INT8), while the MoE experts are compressed to W4A8 (INT4
— GLM-5: from Vibe Coding to Agentic Engineeringto drastically reduce memory footprint without significant accuracy loss
— GLM-5: from Vibe Coding to Agentic EngineeringQuaRot [2] for outlier suppression and Flex_AWQ_SSZ for scaling calibration were employed to maintain stability in low-bit deployment
— GLM-5: from Vibe Coding to Agentic Engineering
Also called W4A8 mixed-precision quantization strategy.
Filed under inference & serving › inference quantization