Retrieval in Pictures: Embeddings, Vector Search and RAG

RAG is short for retrieval-augmented generation. In plain words: look things up first, then answer. The AI searches your documents for useful pieces. Then it writes an answer using those pieces. This lesson shows how the search works, with the key ideas: embeddings, vector search, chunking and reranking.

The big picture

Picture a helpful librarian. You ask a question. The librarian finds the right pages, then explains them. RAG is the same. The search finds the pages. The model writes the explanation.

Embeddings: a map of meaning

A computer cannot compare meaning directly. So it turns text into a list of numbers called an embedding. Texts with similar meaning get similar numbers. Imagine a giant map. Each text is a dot. Dots about refunds sit near each other. Dots about shipping sit in another corner.

• A question becomes a dot on the map too.
• The nearest dots are the most related pages.
• This works even when the words differ. A search for cancel a plan can find a page called ending your subscription.

Vector search

The stored dots live in a vector database. It is built to find the nearest dots fast, even among millions. The word vector just means a list of numbers.

Chunking

Long documents are cut into smaller pieces called chunks. Each chunk gets its own dot. If chunks are too big, they mix topics. If they are too small, they lose meaning. A good chunk is about one idea.

Reranking and hybrid search

• Reranking: the first search returns, say, twenty chunks. A second step sorts them by how well they answer the question. The best few go to the model.
• Hybrid search: combine meaning search with plain keyword search. Keywords are good for names, codes and exact phrases.

Where RAG goes wrong

• The right page was never found, because of bad chunks or old documents.
• The page was found but ignored, because too much text was added.
• The question has no answer in the documents, and the model guesses.
• The documents are stale. Keep them fresh.

A word on knowledge graphs

A knowledge graph stores facts as dots and links, such as person works at company. It helps with questions about relationships. Some systems combine it with RAG. Start with plain RAG first.

1 Prepare   cut documents into chunks, turn each into an embedding, store them
2 Ask       turn the question into an embedding
3 Find      get the nearest chunks, rerank, keep the best few
4 Answer    give the model the question and those chunks
5 Check     show the sources, and say I do not know when nothing fits

A quick example

A staff handbook has two hundred pages. A new hire asks how many leave days she gets. The system turns the question into an embedding. It finds the three nearest chunks. One of them is the leave policy. The model reads that chunk and answers, with a link to the page.