Retrieval-augmented generation (RAG)

Retrieval-augmented generation (RAG) is a technique in which a language model answers in two stages: a retriever first finds the most relevant passages in a document collection, and the model then writes its answer from those passages, ideally citing them. The term comes from a 2020 paper by Patrick Lewis and colleagues describing models that "combine pre-trained parametric and non-parametric memory for language generation," with "a dense vector index of Wikipedia, accessed with a pre-trained neural retriever" (Lewis et al., 2020, NeurIPS 2020).

RAG appeals to compliance teams because it grounds the model's answers. Instead of answering from whatever the model absorbed in training, a RAG system answers from a defined corpus, such as the official text of a regulation, supervisory guidance or the firm's own policies and obligations, and each answer can point to the passage it used. That makes answers checkable and keeps them current as the corpus is updated.

It is not a cure for hallucination. A preregistered study of commercial legal research tools built on retrieval found hallucinated answers on 17% to 33% of queries (Magesh et al., 2025), with causes that included naive retrieval, reliance on inapplicable authority and reasoning errors. Regulatory text adds its own traps: a consolidated version and the original act carry different dates, a repealed provision can linger in the index, and a retrieved passage may come from the wrong jurisdiction. Useful tests are to ask about a provision that does not exist and expect "not found," to check that every citation resolves to a passage that says what the answer claims, and to confirm that the corpus records the version and date of each document, the source provenance behind every answer. RegWatch's Compliance Assistant follows this pattern, answering from an organization's alerts, obligations and policy text, plus any attachments or web search it is given, with structured citations. The LLM accuracy article covers the controls in detail.