Quantization
Nicemleme
D2 · Generative AI, capabilities and limitations
In model compression, quantization represents model values with fewer bits or a reduced set of numerical values.
Review status: 2026-12-05
Technical explanation
Weight quantization can reduce the memory needed to store parameters; GPTQ is one post-training method.
Conceptual boundaries
Fewer bits per weight is different from fewer parameters. This entry concerns model values, not every use of quantization in data processing.
Provider-neutral example
Illustrative: an engineer compares a lower-bit model with its original on the same task and device.
Limitations
Accuracy may decrease; compression alone does not establish latency or quality on a particular device.
Related concepts
Atomic claims and evidence
1.1Google Machine Learning Glossary: quantization
- Source
- Google Machine Learning Glossary: quantization
- Source role
- Authoritative source
- Exact locator
- quantization: reducing bits for parameters; possible decrease in prediction correctness
- Supported claim
- Google describes model quantization as reducing bits used for parameters and notes possible losses in prediction correctness.
- Last verification
- Review due
- Scope limitation
- This uses the parameter-compression meaning of an overloaded term.
2.1GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Source
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- Source role
- Authoritative source
- Exact locator
- Abstract: weight quantization to 3 or 4 bits and evaluated compression results
- Supported claim
- GPTQ evaluates post-training weight quantization at low bit widths.
- Last verification
- Review due
- Scope limitation
- Its measured results do not apply automatically to every model or hardware configuration.