general family · filed under inference & serving
Quantization
A broad family of methods that reduce numerical precision to reduce model memory use and potentially accelerate inference.
Also called quantization algorithms, quantized versions.
- sources
- 3
- models
- 2
- labs adopt it
- 2
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
3 spans quoted from the sources, strongest treatment first.
Quantization is a key technology to run models on consumer hardware by reducing their memory footprint while also accelerating decode speed.
quantized versions run natively on consumer GPUs to power your IDEs, coding assistants and agentic workflows
AngelSlim supports a comprehensive suite of compression tools ... including common quantization algorithms, low-bit quantization
Filed alongside
Other methods under inference & serving :: inference quantization.