Technique
Quantization
Storing model weights in fewer bits so a model runs faster and in less memory.
Part of LLM inference and costQuantization
Connections to other concepts
- Part of
- LLM inference and cost
Practise Quantization
- Advanced Generative AI Interview Questions1 question on Quantization