RAG lab: retrieval-augmented generation with a vector store and a stand-in model
This lab gives each student retrieval-augmented generation (RAG) of their own to build and test: how a program finds the passages that answer a question and sends them to a language model with their sources. They cut documents into chunks, turn the chunks into embeddings, keep them in a small vector store, search it, put what it finds in the prompt, measure the search with recall@k, and see what goes wrong.
The model is a stand-in, and the lab says so everywhere: it answers in the same format as a real model API, but it chooses every reply by fixed rules, so the same question always gets the same answer. Its embeddings are a stand-in too, a hashed bag of words, so two texts are close only when they share words; a real embedding model places texts by meaning. Nothing is downloaded, no AI service is called and nothing costs money. Every copy has no route to the internet and none to any other student.
What students learn
How a chatbot answers from documents it has never been trained on. They cut a page into chunks of two sizes and see a small chunk split a fact in two. They compare embeddings: 0.866 for two texts sharing words, 0.0 for "laptop" and "notebook computer", which a real embedding model would place close. They search 8 chunks from 5 documents, put the closest in a prompt with each source in square brackets, and get an answer that cites it, or none when the passage is missing. They measure recall@1 at 0.75 and recall@3 at 0.875 over eight test questions.
One document hides an instruction, and the stand-in obeys it, which is what a prompt injection is. Setting aside passages that give orders brings back the right answer, and the lesson says what real systems add: treating found text as data, giving the model no power to act, and checking sources. It is framed as a defence on the student's own copy. It suits a course on software development or applied AI; check your own awarding body's unit for the exact wording, none is quoted here.
Run it in your browser
The lab starts two services. model is the stand-in, answering at http://model:4000/v1 inside the lab, with an embeddings endpoint of its own. tools is a Python with rag.py (chunking, the vector store, search and the prompt, written with Python's standard library), five short documents in docs/ and scripts in steps/. The terminal beside the lab has curl and no Python, so it reaches the scripts with docker exec tools.
- Open the Container page. Open Build on the bar and choose Run a container. Anybody may open it, and the Git lab and the SQL lab start with no account. Every other lab needs a free account: create one, sign in, then press Start.
- Find the lab. The Sample labs card lists every sample. Find RAG: search your documents, then answer.
- Press Start. Start runs it as it is, in one press. Edit the file first opens the compose file so you can change it before you start.
- Open it. On the lab's page, press Open terminal and type docker exec tools python3 steps/search.py to see the closest chunks to two questions, with their scores and sources.
What to ask students to hand in
A short document of their own added to docs/, two test questions for it added to steps/recall.py, the recall it printed, and a sentence on what a real embedding model would change. Nothing is marked automatically: the sentence shows whether it was understood.
A basic lesson on this lab
RAG Basics is a small lesson worked in this lab. In it a student chunks a document, compares embeddings, searches the vector store, puts what it finds in the prompt with its source, measures recall@k and stops a planted instruction.
Free personal accounts have it in the Basics module, which is on Home, and every college can copy it from the Library's Basics folder. It is deliberately small: it shows a teacher what a lesson worked in a lab is, and the lab does the rest. Copy it into a module of your own to change it.
The code on GitHub
The compose file has no Dockerfile: it runs one stock public image, Python 3.12 on Alpine, pinned to an exact tag, twice, each as an unprivileged account with 64 MB of memory. The image is under the PSF licence and is not distributed by us. The file writes the stand-in, rag.py, the documents and the scripts from fixed text when it starts, so nothing is downloaded. The vector store is a list in Python, not a vector database: eight chunks need no index. Nothing is kept when the lab stops, so a student copies their work out first.
Set it as work
Say in the task what to hand in.
- Make the lab. In your module add a task and choose the Container lab card, then In a GitHub repository. Give https://github.com/mitsurutech/lab-rag-lab with the branch main. The compose file is the default, compose.yaml. Fork the repository first if you want to change it, for example to put documents of your own in docs/.
- Publish and try it. Publish pins the commit you were shown, so a later change in the repository does not alter a lab you have already set. Start your own copy first.
- Hand it out. Assign it like any task, with a due date if you want one, and say in the task what to do and what to hand in.
- Mark what is handed in. Open a student’s copy and the work they handed in, mark it, and send it back with a reason if it needs more.
Try this lab, and what it costs
A free personal account can work through the basic lessons in this lab, with enough free credit each month to follow most lessons and no card. For a class, a school or college has its first 365 days free for up to 50 students, then pays per student a year, with every lab included and staff free.
Questions people ask
- Are the embeddings real?
- No. The stand-in hashes each word to a place in a list of numbers, so two texts are close only when they share words. A real embedding model is trained to place texts by meaning, so "laptop" and "notebook computer" would be close and the notebook question would be found. Chunking, the store, the prompt and recall@k work the same way with either.
- Is there a real AI model in this lab?
- No. The model is a stand-in written for the lab, a short Python program that chooses each reply by fixed rules. It answers only when the passage holding the fact is in the prompt, which is how it shows what missing context does. It always gives the same answer to the same question.
- Why does the lab include a document with a hidden instruction?
- To teach the defence. Text found by search is somebody else's writing, and a model may follow instructions in it. Students see that happen on their own copy, then stop it. The lesson asks them to try it only on data that is theirs.
- Does it cost anything or use credits for AI?
- No AI is used, so nothing is charged for it. The lab runs like any other container lab.
- Why a list in Python and not a vector database?
- Eight chunks are searched in a moment by comparing each one. A vector database adds an index so that millions can be searched quickly, which matters in a real system and would hide the idea here.