An often mentioned problem with GPT models such as ChatGPT is the “hallucination” issue. Luckily, there is also a solution for it: RAG, short for Retrieval-Augmented Generation.
Let’s break down how RAG helps reduce “hallucination” in AI, using simple terms and relatable examples.
What is Hallucination in AI?
Hallucination in AI happens when a model generates information that seems plausible but isn’t actually true or supported by any data. Imagine if you asked a friend about a movie you both watched, but they started telling you about scenes that never happened. That’s kind of what AI hallucination looks like.
Why are GPT models hallucinating?
Because they lack meta-knowledge.
An example I often use: if I ask you who was the mayor of the French capital Paris in 1964, what would you say?
“Uhm, I don’t know.”
Because typically these are things you don’t know. You don’t even try to think: you immediately know that you don’t know. That’s metaknowledge: knowledge about your own knowledge.
That also explains why humans normally do not hallucinate, but GPT models do. Because their architecture is different. Classic GPT models are “word predictors”: they just use a very advanced pattern matching algorithm to predict the next word in a sequence. (More about GPT architectures in another blog post)
So they do not “think” about the question. They just “utter words” like people playing domino, trying to find the best match in their library of words.
Now as I mentioned before, RAG or Retrieval Augmented Generation solves this problem.
How Does RAG Work?
RAG combines two main steps:
- Retrieval: This step is like looking up facts in a library before answering a question. The AI searches through a vast database of documents or information to find pieces that are relevant to the query you’ve made.
- Generation: Once it has these relevant pieces of information, the AI uses them to craft an answer, much like a student would use notes to write an essay.
Solving hallucination
How does RAG solve the hallucination problem? There are three aspects that make its architecture superior to a raw GPT:
- Access to real information: By having a library of facts to draw from, RAG reduces the chance of making up information because it’s anchored to real data. For example, if you ask about the capital of France, instead of guessing, RAG would find documents stating “Paris is the capital of France” and then use that information. If the documents do not contain information about Paris and France, the RAG will say so (and might then rely on its GPT-capability to “guess” an answer, but at least you are warned that it might be hallucinating)
- Contextual understanding: RAG doesn’t just find any information; it looks for information that fits the context of the question. If you ask about the diet of a panda, RAG would retrieve information specifically about panda diets, not about their habitat or behavior, ensuring the answer is directly relevant.
- Verification: Since RAG uses real documents, there’s a kind of built-in fact-checking. If the AI starts to generate something that conflicts with the retrieved data, it can self-correct.
Example:
Imagine you ask an AI, “What did Albert Einstein invent?”
Without RAG: The AI might say, “Einstein invented the light bulb.” This is a hallucination because Einstein is not associated with the invention of the light bulb.
With RAG: The AI retrieves information about Einstein, finds mentions of his contributions to theoretical physics, like the theory of relativity, and then responds, “Albert Einstein is famous for his theory of relativity, not for inventing physical devices like the light bulb.”
Benefits of RAG
RAG systems allow you to access data in structured or unstructured format via classic GPT models such as ChatGPT, Gemini or Claude. So on top of the GPT ‘interface’, you get additional benefits:
- Trustworthy information: You get answers that are more likely to be correct because they’re based on actual data.
- Educational: It can serve as a learning tool, providing accurate context and references for further exploration.
- Reduced misinformation: In an era where accurate information is crucial, RAG helps in cutting down the spread of false information.
In essence, RAG is like having a smart assistant who always checks the facts before speaking, ensuring what you hear is closer to the truth. I use a (self-trained) RAG all the time. I train it by uploading documents (PDF, word, excel, powerpoint, text, even audio) to it. If you want to give CoralAI a try, you can do so for free (you only have to pay if you want to upload multiple documents).