Chatbot service lab: the program between a chat window and a language model
This lab gives each student a small chat service of their own: the program that sits between a chat window and a language model and does the work the model does not. It keeps each conversation, trims it to a budget of tokens, applies a system prompt and a persona, checks words on the way in and out, streams replies, copes with a busy or broken model, limits each user and keeps a log with no personal data. It is tested with fixed transcripts.
The model is a stand-in, and the lab says so everywhere: it answers in the same format as a real model API, but it chooses every reply by fixed rules, so the same question always gets the same answer. That is what makes a transcript a test. Nothing is downloaded, no AI service is called and nothing costs money. Every copy has no route to the internet and none to any other student.
What students learn
What a chat service adds around a model. They see a name remembered in one session and unknown in another, because the service, not the model, keeps each conversation. They watch a context budget decide what is sent: 85 tokens of history keep the name, 37 lose it. They see a persona refuse a question, a rude message never sent, and an email address and phone number removed from a reply. They stream a reply a checked sentence at a time, see a busy model answer on its third try and a broken one give the user an apology with status 503, and run 9 checks over fixed transcripts. It suits a course on software development or applied AI; check your own awarding body's unit for the exact wording, none is quoted here.
Run it in your browser
The lab starts two services. model is the stand-in, answering inside the lab. chat is the chat service, a Python file written with the standard library, answering at http://chat:8000/chat, with its settings at the top of /tmp/chatbot.py. The terminal beside the lab has curl and no Python, so it calls the service directly and runs the service's own scripts with docker exec chat.
- Open the Container page. Open Build on the bar and choose Run a container. Anybody may open it, and the Git lab and the SQL lab start with no account. Every other lab needs a free account: create one, sign in, then press Start.
- Find the lab. The Sample labs card lists every sample. Find Chatbot service: sessions, moderation and limits.
- Press Start. Start runs it as it is, in one press. Edit the file first opens the compose file so you can change it before you start.
- Open it. On the lab's page, press Open terminal and send a message with curl, as the lesson shows, to the address http://chat:8000/chat. Then type docker exec chat python3 test_transcripts.py to run the tests.
What to ask students to hand in
One change to chatbot.py, such as a word it refuses, a new persona, a different budget or a stricter limit, with a transcript in test_transcripts.py that proves the change works. With it, the test output and a sentence on what a real model would do differently. After a change, docker restart chat starts the service again and keeps the edited file.
A basic lesson on this lab
Chatbot Service Basics is a small lesson worked in this lab. In it a student keeps a conversation in a session, sees the context budget at work, tries a persona, checks words in and out, streams a reply, retries a busy model and runs the tests.
Free personal accounts have it in the Basics module, which is on Home, and every college can copy it from the Library's Basics folder. It is deliberately small: it shows a teacher what a lesson worked in a lab is, and the lab does the rest. Copy it into a module of your own to change it.
The code on GitHub
The compose file has no Dockerfile: it runs one stock public image, Python 3.12 on Alpine, pinned to an exact tag, twice, each as an unprivileged account with 64 MB of memory. The image is under the PSF licence and is not distributed by us. The file writes the stand-in, the chat service and its tests from fixed text when it starts, with Python's standard library alone, so nothing is downloaded. Nothing is kept when the lab stops, so a student copies their work out first.
Set it as work
Say in the task what to hand in.
- Make the lab. In your module add a task and choose the Container lab card, then In a GitHub repository. Give https://github.com/mitsurutech/lab-chatbot-lab with the branch main. The compose file is the default, compose.yaml. Fork the repository first if you want to change it, for example to give the chat service another persona or a stricter limit.
- Publish and try it. Publish pins the commit you were shown, so a later change in the repository does not alter a lab you have already set. Start your own copy first.
- Hand it out. Assign it like any task, with a due date if you want one, and say in the task what to do and what to hand in.
- Mark what is handed in. Open a student’s copy and the work they handed in, mark it, and send it back with a reason if it needs more.
Try this lab, and what it costs
A free personal account can work through the basic lessons in this lab, with enough free credit each month to follow most lessons and no card. For a class, a school or college has its first 365 days free for up to 50 students, then pays per student a year, with every lab included and staff free.
Questions people ask
- Is there a real AI model in this lab?
- No. The model is a stand-in written for the lab, a short Python program that chooses each reply by fixed rules. It answers in the same format as a real model API, so the chat service would work against a real one by changing one address, but its words are not generated. It always gives the same answer to the same question, which is what lets a fixed transcript be a test.
- How is this different from the LLM workbench lab?
- The LLM workbench is the model's API itself: requests, roles, tokens and streaming. This lab is the service a real chatbot puts in front of that API: sessions, budgets, personas, moderation, retries, limits, logs and tests. Either can come first.
- Does it cost anything or use credits for AI?
- No AI is used, so nothing is charged for it. The lab runs like any other container lab.
- What does the chat log keep?
- One line a turn in /tmp/chat.log inside the chat service: the time, the user and session as short hashes, token counts, how many tries it took and whether anything was refused or removed. Never the words, so no name or message can leak from it. The tests check that.
- Can my students send anything personal to it?
- It goes nowhere: the stand-in runs inside the student's own copy, which has no way out to the internet. It is still a good habit to teach that a real model service receives whatever is typed, so nothing personal or secret belongs in a message.