RAG overview
When pasting stops scaling: retrieval-augmented generation in plain terms — embeddings, chunking, and the retrieve-then-ground pipeline.
Grounding works beautifully until the document set outgrows the prompt. One policy page pastes fine. Two hundred help-center articles, a product catalog, and three years of order FAQs do not — and even inside a giant context window, cost and attention-dilution punish you for shipping the whole library on every call. The scaling answer: retrieve first, then ground. That's Retrieval-Augmented Generation — RAG.
The pipeline in four moves
- 1Chunk — split the library into retrieval-sized pieces (a section, a paragraph — typically a few hundred tokens each), because you want to fetch the relevant part, not whole documents.
- 2Index — convert every chunk to an embedding: a list of numbers positioned so that texts with similar meaning sit near each other. 'Can I send this back?' lands near the returns section despite sharing zero keywords.
- 3Retrieve — embed the incoming question, find the nearest chunks (top 3–10), and pull them.
- 4Ground — build the prompt you already know how to build: instructions +
<context>containing the retrieved chunks + closed-book rules + the question.
Here's the reframe that makes this course's skills transfer: RAG is context injection with a search engine bolted on the front. Everything from the last lesson — the closed-book rule, the honest exit, fencing, positioning — applies verbatim to retrieved chunks. The new failure surface is the retrieval itself.
The failure mode that fools everyone
When a RAG system answers wrong, teams instinctively tune the prompt. But the most common cause is upstream: the right chunk was never retrieved — so the model, holding only wrong context and a closed-book rule, did its best with what it got. Debugging discipline: always look at the retrieved chunks before touching the prompt. If the answer isn't in them, no prompt fixes it — you have a chunking or retrieval problem.
- Chunk too small → retrieved text lacks surrounding context ('the fee is waived' — which fee?).
- Chunk too big → relevant sentence buried in noise, attention diluted.
- Question vocabulary ≠ document vocabulary → embeddings usually bridge it, but jargon-heavy corpora may need query rewriting.
This lesson makes you a competent RAG consumer and designer — you can whiteboard the pipeline and debug the prompt half. Building one for real (vector stores, chunking strategy, retrieval metrics, re-ranking) is the entire Building RAG Applications course, which follows this one in three of the tracks.