Grounding the Answer

Injecting the context

Retrieval handed you the best passages; now you assemble the prompt. Context injection means placing those passages into a prompt template alongside the user's question and a clear instruction about how to use them. A typical layout: a system prompt setting the rules, then the retrieved chunks (each tagged with its source), then the question.

SYSTEM:
  Answer the question using ONLY the context below.
  If the answer is not in the context, say "I don't know."
  Cite the source tag [S1], [S2]... after each claim.

CONTEXT:
  [S1] (handbook.pdf p.12) Refunds are issued within 14 days...
  [S2] (faq.md) Refunds require the original receipt...

QUESTION:
  How long do refunds take, and what do I need?

A grounded prompt: rules first, tagged evidence next, question last.

Instructing the model to stay grounded

Handing over passages isn't enough; you must tell the model to use them and only them. Grounding is enforced with explicit instructions: answer only from the context, and critically, if the answer isn't there, say you don't know. That permission to abstain is the single most effective guard against hallucination — without it, the model fills gaps by guessing.

P(\text{answer}\mid q,\,\mathcal{C}) = \prod_{t} P\!\left(y_t \mid y_{<t},\, q,\, \mathcal{C}\right)

Grounding conditions every generated token on the query q and the retrieved context C — the answer is a function of the evidence.

Citations and attribution

Because you tagged each chunk with its source, you can ask the model to cite — attribution turns an answer into something a user can verify. Footnote-style markers like `[S1]` after each claim let a reader click straight through to the original passage. Trust in a RAG product comes far more from checkable sources than from a confident tone.

One trap to watch: a model can produce a fabricated citation — a real-looking reference to a document that doesn't say what's claimed, or doesn't exist. Defend against it by only allowing citations to the retrieved chunks (with their real ids), and verify after generation that every cited id was actually in the context. Never let the model invent the bracket label.

Failure modes to expect

Two recurring problems. First, lost in the middle: long prompts get the most attention at their start and end, so a key passage buried in the centre can be overlooked. Rerank well and keep only a few chunks so the important one sits near an edge. Second, context overflow: pack in too many chunks and you blow the context window or drown the signal. More retrieved text is not better — the right few beat the many.