RAG
Let the model look things up in your documents before it answers.
RAG (Retrieval-Augmented Generation) is a way to make an LLM answer from documents it never saw during training: your company's manuals, last week's contracts, a product catalog. Instead of retraining the model, you fetch the relevant pages at question time and paste them into the prompt. The model then writes its answer with those pages in front of it.
It works in two steps. First, retrieval: the documents are split into chunks and each chunk is turned into embeddings, numbers that capture its meaning, stored in a database. When a question arrives, the system finds the chunks closest in meaning to it. Second, generation: the question plus those chunks go to the LLM, which answers and can say where it found each thing.
Why it matters: an LLM's knowledge is frozen at training time, and it knows nothing private. RAG fixes both without the cost of training, keeps the data where it already lives, and makes hallucinations rarer because the model is grounded in real text. It is the most common architecture behind 'chat with your documents' products and internal company assistants.
Example: a support bot for an airline. The question 'can I bring my cat?' retrieves the pet policy page, and the model answers from that page instead of guessing what airlines usually do.
Keep in mind
- RAG adds knowledge at question time, through the prompt, not by retraining.
- Retrieval usually relies on embeddings and a vector search.
- Answers can cite their sources, which makes them easier to check.
- Quality depends on retrieval: if the wrong chunks come back, the answer is wrong too.
