What Is RAG? How AI Learns to Look Things Up

[00:00:00]There are two ways AI can answer your question. It can remember or it can look. For most of AI's history, it could only remember. Rag is what changed that.

[00:00:12]Rag or retrieval augmented generation is how an AI finds the right information for your specific question. It reads that information before answering instead of working from whatever it happened to learn during its training.

[00:00:27]Here's what the problem looks like without it. In 2024, Air Canada's chatbot told a grieving passenger that he had time after booking to apply for a bereavement discount. He booked the flight, then he applied. The policy actually didn't allow it, and a court agreed with him, not the chatbot. The bot wasn't lying. It was answering from information that was accurate when it was trained. The policy had since changed, and the chatbot had no way to know that. When AI is trained, it reads an enormous, enormous amount of text, web pages, articles, documentation, books. It learns how language works, how concepts connect, how to think over a question. And then the training just ends. The moment training ends, the model's knowledge freezes. Everything after that date doesn't exist to it. New policies, updated prices, recent events.

[00:01:24]Not because it forgot, it never had the chance to know. Studies of realworld AI interactions find that around 31% of responses contain some form of hallucination. The danger isn't that the model is sometimes wrong. It's that it's wrong in the exact same confident voice it uses when it's right. Air Canada's chatbot wasn't an outlier. It was a closed book exam taker answering from memory and the memory was old. So what's the fix? What if the model didn't have to rely on memory at all? What if right before it answered, something handed it the relevant documents and said, "Here, read these first." That's rag. Before the model generates an answer, a retrieval system searches a knowledge base, current policy documents, a product catalog, a database of recent updates. It pulls out all the most relevant pieces and then hands them over to the model alongside your question.

[00:02:22]Now, the model doesn't have to remember the bereavement policy. It can just read it immediately from a source that was updated this morning if it needed to be.

[00:02:32]Let's break it down in three simple steps. First, your question comes in.

[00:02:38]Second, the system ranks everything in the knowledge base by how relevant it is to your question and pulls the top results. Not the entire policy manual, just what ranked closest to what you asked. Third, those results land in front of the model alongside your question. It reads the source, then it answers. The retrieval is what changed everything. The model isn't reaching into its own memory. It's reading a source that you control. Rag doesn't just fix one problem. It fixes an entire class of them. Stale information, missing information, your own data that the model was never trained on, and hallucination, which drops substantially when a model reads actual source text instead of reconstructing it from memory. You also control what it reads, which means that you can update it, correct it, and trace any answer back to its original source. What it doesn't fix is the source material. So, if the source is wrong, the answer is going to be wrong. If retrieval pulls the wrong chunk, the model confidently grounds in the wrong thing. An open book exam does not guarantee a good answer. It just removes the excuse for not having the information. But here's what it does change. The search step that makes this work, finding meaning instead of matching keywords, is called embeddings.

[00:03:58]But that is its own episode. Circling back to Air Canada for a sec, a chatbot with rag reads from the live bereavement policy web page, updated whenever the policy changes. It reads the current rule before it answers, not the rule from 18 months ago, the one from this very morning. And it answered correctly, not because it remembered, because it looked.