Back to course overview
Module 4Grounding with context & RAG basics 12 min

Context injection

Grounding: making the model answer from documents you provide instead of its training memory — the closed-book rule and the context budget.

Your triage assistant has been running on the model's general knowledge plus your rules. But real questions need your facts: Harbor Lane's return policy, current shipping times, the product catalog. Grounding means supplying those documents in the prompt and — critically — making the model answer from them rather than from its training-data memories.

The closed-book instruction

Pasting the policy in isn't enough; the model will happily blend your policy with its statistical memory of 'typical retail policies' — producing answers that are 80% yours and 20% invented industry average. The fix is an explicit epistemic rule:

grounding-block.txttext
<policy>
{Harbor Lane return & shipping policy text}
</policy>

GROUNDING RULES
- Answer questions about policy, shipping, and returns ONLY from <policy>.
- If the answer is not in <policy>, say so in the reply and set confidence LOW
  — never fill gaps from general knowledge.
- When the reply states a policy fact, it must match <policy> exactly
  (durations, amounts, conditions).

That middle rule is the load-bearing one. Without a legal way to say 'not in my documents', the model treats every question as answerable — and the gap-filling is invisible because it sounds exactly like the grounded parts. Same principle as the schema's null policy: always give the model an honest exit.

Managing the context budget

  • Only load what the task needs. The triage assistant needs the returns policy, not the employee handbook. Every irrelevant page dilutes attention (Module 1) and costs money.
  • Structure documents before pasting — headings and short sections survive; walls of text degrade. Ten minutes cleaning a policy doc pays back on every call.
  • Position by priority: instructions at top, documents in the middle, the task + 'answer from the documents above' restated at the end — the edges are where attention lives.
  • Version your context like your prompt. 'Which policy version was live when the model said that?' is a question you will eventually be asked. Date-stamp inside the fence.
Prompt to try

Draft a one-page returns-and-shipping policy for Harbor Lane (invent reasonable terms: 30-day returns, damage exceptions, holiday extension). Then I'll paste it into my triage prompt as <policy> with grounding rules, and test with: (1) a question the policy answers, (2) a question it half-answers, (3) a question it doesn't cover. Predict which of the three will tempt the model to improvise.

It's (2) every time — total gaps trigger the honest exit, but partial coverage invites blending. Your eval set needs a half-covered case more than it needs ten clean ones.