Skip to content

RAG explained simply: an assistant that answers from your own content

RAG lets an AI assistant answer from a business's own content instead of its memory. How it works step by step, where it falls short, and good habits.

Knowledge and RAG5 min read

A large language model can write, rephrase and summarise. What it does not know is the opening hours of your client’s bakery, the return policy of their online shop or what the manual of their best-selling product says. If it answers anyway, it may produce a sentence that sounds right and is wrong.

RAG is the technique that closes this gap. It stands for retrieval-augmented generation. The term comes from a 2020 research paper by Patrick Lewis and co-authors, listed at the end of this article. The idea is simple: before answering, the assistant looks up the useful passages in a set of documents, then writes its answer from those passages.

This article walks through how it works, step by step and without maths, so that you can explain it to a client in turn.

The open-book exam

Picture two students. The first sits a closed-book exam: they answer from what they remember and fill the gaps as best they can. The second sits an open-book exam: before each answer, they find the right page in the textbook and work from it.

A language model on its own is the first student. A RAG assistant is the second. The textbook is your client’s content. Answer quality therefore depends less on the model’s general knowledge than on the quality of the textbook and on finding the right page.

The steps, one at a time

RAG happens in two phases: preparation, done once and then after each update, and retrieval, done for every question.

1. Read the content

The tool collects the content to use. In blabla.bot, three kinds of sources are available today: the address of a web page (crawling stays within the same site), a sitemap listing the pages to read, and PDF files, read page by page. Pages excluded by the site’s robots.txt file or marked “noindex” are skipped, and the console tells you why each skipped page was left out.

2. Split into passages

A whole page is too long to be useful as it is. The content is split into passages of a manageable size. Each passage remembers where it came from: the page title and address, or the file name and page number for a PDF.

3. Turn each passage into a vector

This is the most abstract step. A specialised model, called an embedding model, turns each passage into a long list of numbers called a vector. Two passages about the same thing get vectors that are close together, even when they use different words. “What are your opening hours?” and “Are you open on Sundays?” end up as neighbours.

4. Find the passages closest to the question

When a visitor asks something, the question is turned into a vector the same way. The system then looks for the passages whose vectors are closest. Each passage gets a relevance score. In blabla.bot the number of passages kept depends on the AI option: 4 in Simple mode, 8 in In-depth mode.

5. Write the answer from the passages

The selected passages go to the language model together with the question and the agent’s instructions. The model’s rule is to answer from those passages and to cite the ones it used. Below the answer, the visitor sees the cited sources, and only those.

A concrete example

Take a car repair garage’s website. A visitor types: “Do you service air conditioning?”

StepWhat happens
RetrievalThe system finds a passage on the “Our services” page that mentions air conditioning recharge, and one on the “Prices” page
ScoreBoth passages are above the relevance threshold
WritingThe model answers that air conditioning recharge is one of the services and points to the prices page
SourcesThe widget shows both cited pages, with their links

If the visitor asks “Do you sell used cars?” and the site says nothing about it, no passage reaches the threshold. The agent then says it does not have that information, instead of inventing it.

Why it beats a model on its own

  • Grounded answers. The agent works from specific passages, not from general memory.
  • Checkable answers. The sources shown let visitors verify.
  • Simple updates. To change an answer, you edit the page on the site and re-index the source. No model retraining is needed.
  • Separation between agents. In blabla.bot every search is filtered by agency and by agent: one agent never sees another client’s knowledge.

Limits worth knowing

RAG is not magic. Three limits come up again and again.

Content quality. If a page is vague, contradictory or out of date, so is the answer. RAG faithfully reproduces what you feed it.

Text only. A scanned PDF with no text layer cannot be read, and blabla.bot reports it with an explicit error. Images, videos and tables saved as pictures are not used.

Tuning the threshold. A threshold that is too low lets off-topic passages through. One that is too high makes the agent say “I don’t know” too often. In blabla.bot the threshold is calibrated for the embedding model in use, because a score only means something for a given model.

There is also a cost. Turning content into vectors, and then every question, uses tokens at the embedding model’s provider. In blabla.bot that cost is billed like everything else, at provider cost times two, and indexing shows an estimate before it starts.

Key takeaways

RAG turns a general-purpose language model into an assistant that knows one specific business. It works in two phases: prepare the content as indexed passages, then, for each question, find the right passages and answer from them. Its strength rests on the quality of the content and on its ability to say “I don’t know” when nothing matches.

To put this into practice, the next article explains how to prepare a knowledge base. Common questions about how blabla.bot works are gathered in the FAQ.

Source

Tags

  • RAG
  • knowledge base
  • embeddings
  • vector search