Fine-Tuning Small Language Models
Decide when to fine-tune a small model vs. use a frontier model, choose the right technique (LoRA, distillation, quantization), and design a hybrid routing architecture for production
4 sections ยท 11 lessons
Course outline
Why Small Models Matter in Production
- The cost and latency case
- Data privacy as a forcing function
- When small beats frontier
Fine-Tuning Techniques
- LoRA: fine-tune in an hour
- Distillation: learning from a larger model
- Quantization: shrinking model size
- RAG vs. fine-tuning: when to use each
Limitations and Failure Modes
- Where small models still fail
- Catastrophic forgetting
Production Architecture
- The 90/10 routing pattern
- Speculative decoding