LLMs & Generative AI
Pretraining, Fine-Tuning & Instruction Tuning
Pretraining learns broad capability from massive data; fine-tuning specializes behavior; instruction tuning and preference optimization shape the model toward useful human-facing responses.
- Pretraining creates general language capability
- Fine-tuning changes weights for a narrower distribution
- Instruction tuning teaches task-following
- Preference optimization ranks outputs by human preference
- RAG is often better than fine-tuning for changing knowledge
| Need | Usually prefer | Why |
|---|---|---|
| Add current/private facts | RAG | Knowledge stays inspectable and updateable |
| Enforce strict shape | Structured output + validation | Format can be checked in code |
| Teach stable style/domain behavior | Fine-tuning | Examples become behavior |
| Improve instruction-following | Instruction tuning / prompt design | Task behavior becomes clearer |
| Improve subjective preference | Preference optimization | Learns ranking of outputs |