RAG

RAG (Retrieval-Augmented Generation) is a method where AI reads relevant material before answering a question. It works in three steps — when a question comes in, the system retrieves relevant passages from documents or databases, then generates an answer based on that evidence.

The "open-book exam" analogy fits well. Relying on memory alone (closed-book) makes plausible-sounding wrong answers — hallucinations — easy to produce, but letting the model check a textbook first improves accuracy and lets it cite sources.

This is the standard approach companies use to build "AI that knows our internal documents." It's cheaper than retraining the model (fine-tuning), and answers update immediately when the source material changes. Behind stories about "a chatbot that answers company policy questions" or "AI grounded in medical literature," there's almost always RAG at work.

"They adopted a RAG architecture to reduce hallucination" — meaning they made the model look up supporting documents before answering, to boost reliability.

1. Before attaching any document (a product manual, terms of service, internal policy, etc.), first decide what the ideal answer should be yourself — "If someone asks about the refund deadline, what should the answer look like?" Defining the expected output first is rule number one for people who actually build RAG systems.
2. Now attach the document and ask: "Answer based only on what's in this document. If it's not in the document, just say 'not in document.' Question: how many days is the refund deadline?" — Compare this to the ideal answer you defined in step 1.
3. Try setting a trap — deliberately ask about something not covered in the document: "What's the refund deadline for international shipping?" If it says "not in document," that's a pass; if it makes up something plausible-sounding, that's a hallucination.
4. One more step: paste the answer you got into a new conversation and ask, "Grade whether this answer matches the original document text below." This is a mini version of what's called "using an LLM as an evaluator" in practice — using AI to check AI's own answers. Keep a running list of questions and their ideal answers, and rerun it whenever the source document changes — that's essentially what companies mean by a test set.

People use the phrase "we trained the AI on our company data" for both, but the methods are completely different. One retrains the model itself to change its underlying behavior, while the other leaves the model untouched and simply fetches relevant documents to hand over at query time. When a company says "let's build an internal chatbot," what they usually need is the latter.