Learning About RAG

This evening I decided to finally dive into RAG (Retrieval-Augmented Generation) — a topic I didn’t know too much about but had seen in documentation here and there. I knew that it was used to search large amounts of data to answer questions, but I didn’t know how it worked practically.

Lately, I’ve been curious about personal knowledge management (PKM) and how to build a workflow for storing technical research I had done over time (either manually or agentically). RAG seemed like a great fit — mostly because I can have an AI agent query my notes without stuffing my entire directory into its initial context window and blowing up the token limit.

This post isn’t going to teach you about RAG, but rather about my experience learning about it.

Learning objectives

• Don’t dive too deep - I wasn’t going to go and implement a RAG system from scratch, I just wanted to know the core principles on how they work to possibly build tools around them.
• Guided learning - I’m not great at sitting and reading large papers or sites to learn about something; most of the time they’ve got too much information and I end up getting lost. I needed some practical examples, with checkpoints to ensure I was comprehending everything.

Methodology

Like a lot of people these days, if I have a specific learning objective I go to my favourite LLM and ask away.
The choice for me generally is Gemini (because I was already paying for Google One). I used its “Guided learning” in “Thinking” mode to get started on my learning journey. This was my prompt:

Teach me about LLM RAG, what is it and what it’s used for. Provide examples that someone who has surface knowledge on how LLMs work could understand. Ask questions to gauge if I understand the topic

The model came back with an intro to the topic, using analogies that made it super easy to grasp. A high level How it works, a Why use RAG, then finally the start of the learning journey - exploring each of the outlined topics step-by-step and an opening question to make sure I understood its high-level overview.

Things I liked about this approach

Not too much jargon - The balance between technical depth and simplicity was just right for me. It didn’t throw too many technical terms at me at once, but it also didn’t oversimplify to the point of being patronising.

Good analogies/personal context - I had saved in Gemini Memories that I was a Software Engineer, so the model tried to be relatable by giving examples.

Code examples - After establishing some basic terms, the guided learning stepped it up a notch with some Python examples using llama_index to create and query a vector database.

This was particularly helpful. The snippets weren’t too verbose, providing just enough info to get the point across. (Later, I actually used these to go vibe code a working prototype in Antigravity).

Very clear checkpoints - After each topic, the model would ask a question to ensure I understood the topic before moving on. In some cases I would answer but then ask a follow-up question to test my understanding further. For example when it showed me a llama_index script I asked:

In this script, the database is ephemeral. For a large system this would take some time to start up. Is there a way to store this VectorStore and then read from it later, reindexing?

To which it responded:

Absolutely. In a production environment, you wouldn’t re-index your entire knowledge base every time the service restarts—that’s a classic O(N) bottleneck you want to avoid. ⏱️
Instead, we introduce a Persistence Layer. Think of this as the librarian saving their entire card catalog to a hard drive so they don’t have to re-read every book when the library reopens the next morning.

A Classic O(N)! I thought this was a bit funny. It provided me with an answer specific to that Python library and then got back on track with the prescribed learning topic instead of going off on a tangent - nice.

Summarisation for archival purposes

After I’d had enough for the night, I asked it to print out a summary:

I’ve had enough learning for today. So that I can reference this later, provide me with a comprehensive summary of what we’ve talked about and learned today. I will be storing this document for later so any external reference material you used should be linked.

I won’t share its whole output, but if you’re interested in learning about the same topic go and give this method a spin and see if it helps.

Am I an expert now?

No, but I’m a lot more comfortable with the topic and I feel like I have a good basis to build upon. For less than an hour of learning I think it was just the right amount of information that would give me a solid foundation to dive deeper into specific areas later on if I wanted to.