K-Culture GlossaryㅇTechnical words in the news
Quantization
Compression that shrinks a model by lowering the precision of its numbers — the trick that lets LLMs run on phones.
In plain words
Quantization is a compression technique that reduces size and computation by lowering the precision of the numbers inside a model. Instead of storing weights as 16-digit decimals, you store them as 4- to 8-bit integers — similar to lowering a photo's quality a notch to shrink its file size drastically.
The key point is that performance barely drops even after cutting precision. The fact that LLMs can run on laptops and phones, and the community culture of running open models on personal PCs, all exist thanks to quantization. When you see a phrase like "4-bit quantized version released," read it as "now you can run this on your home computer."
Distillation vs. Quantization: What's the Difference?
Lowering the precision of an existing model, Training a new, smaller model
Both get described as 'making a model lighter,' so they look like the same technique. But one means training a separate, smaller model to learn from a larger one's abilities, while the other means shrinking the size of an existing model by coarsening its numbers. They're often combined too — distill down to a small model, then quantize it further.
| Aspect | Quantization | Distillation |
|---|---|---|
| What it does | Reduces the number of digits used to represent the model's numbers | Trains a smaller model using a larger model as a teacher |
| The result | A lighter version of the same model | A new model, distinct from the original |
| What it costs | Just a conversion — much faster | Requires training — time and hardware |
| What's lost | Precision is trimmed, slightly blurring answers | Falls short of the teacher's ability |
| In articles | "A 4-bit version that runs on phones" | "A small model distilled from the large model is released" |
Rule of thumbIf a new model is created, it's distillation; if the same model becomes lighter, it's quantization.
See also
Stories using this term
No story has used this term yet. New ones attach here automatically.