Retrieval-augmented generation, or RAG, searches an external collection and places selected material in the model's context before generation. The model can then answer from evidence that was not encoded reliably in its parameters.
RAG is useful for private documents, current information, citations, and large collections that do not fit in one prompt.
- Prepare offline
Parse and chunk source documents; retain metadata and embed them into an index with access fields.
- Keep provenance
Example metadata records source policy-7, version 2026-09-01, and tenant store-42.
- Retrieve online
Authorize the user’s retrieval scope, retrieve candidates, then rerank and select.
- Generate and cite
Generate from selected evidence and return citations. Check whether the evidence actually supports the answer.
Generation cannot repair missing or unauthorized evidence.
How to read this model
An indexing cabinet beside a filtered retrieval carriage. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
RAG joins two imperfect systems
A RAG answer depends on both retrieval and generation:
question -> retrieve candidates -> select evidence -> generate answer
If retrieval misses the needed passage, generation cannot use it. If retrieval succeeds, the model can still misread the passage or add an unsupported claim. Evaluate the stages separately.
The index is not the source of truth
An index is a derived representation. Documents can be stale, duplicated, partially parsed, or removed without a matching index update. Preserve stable source identifiers, versions, access metadata, and links back to the authoritative record.
For a policy assistant, each chunk might carry:
{
"sourceId": "policy-returns",
"sourceVersion": "2026-09-01",
"section": "Damaged items",
"tenantId": "store-42",
"accessGroups": ["support", "returns"]
}
That metadata supports access filtering and lets a citation point to the exact source version.
A complete pipeline has an offline and online path
The offline path parses sources, splits content, adds metadata, computes embeddings, and updates indexes. The online path interprets a query, applies authorization filters, retrieves candidates, reranks or selects them, constructs context, generates an answer, and returns citations.
Freshness connects both paths. A pipeline needs a rule for updates, deletions, failed indexing, and re-embedding after a model change.
Refusal is a valid result
If retrieved evidence is missing, contradictory, unauthorized, or too weak, the system should say so. Forcing an answer converts an information gap into a confident guess.
The application can define a policy such as:
- Answer only when at least one selected passage supports the claim.
- Cite every externally verifiable claim.
- Report conflicts rather than merging them silently.
- Ask a clarifying question when the query lacks required scope.
These rules still need evaluation. A prompt alone cannot prove compliance.