In plain English
Smaller models are cheaper, faster and can run on your own device. Distillation trains a small student to copy a large teacher on a focused task. Quantization stores a model's numbers with fewer bits. Test on hard and rare cases before you switch.
- Distillation
- A small model learning from a large one.
- Quantization
- Storing a model's numbers with fewer bits.
- Student model
- The small model that copies the teacher.
Picture it
How distillation works
1
Big teacher model
Strong, slow and costly
2
Teacher answers many cases