LLMs & Generative AI

Retrieval-Augmented Generation (RAG)

RAG retrieves external evidence at answer time and gives it to the model as context. It separates knowledge storage from language generation, making answers more current, inspectable, and citeable.
  • RAG is for knowledge outside the model
  • Retrieval quality dominates answer quality
  • Chunking defines the unit of memory
  • Reranking improves precision after broad retrieval
  • Citations require source tracking
  • RAG does not guarantee truth
RAG pipeline
Every step can be evaluated separately.
RAG failure modes
FailureSymptomFix
Bad chunkingRelevant answer split or missing contextChunk by sections, include headings
Low recallCorrect source not in top-kHybrid retrieval, better embeddings
Low precisionContext full of distractorsReranking, filters, metadata
Stale sourceAnswer uses old policyFreshness filters, source versioning
No grounding checkAnswer goes beyond evidenceRequire citations / abstain behavior
Sources
  • Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Retrieval-Augmented Generation
  • OpenAI API Documentation — File search and retrieval
  • Hugging Face LLM Course — Using Transformers and datasets