What does an app send to a chatbot?
A homework helper or a shop's chat window is a program. It sends your words to a language model and shows what comes back. In this lab you write that program's side: the request, the answer and what goes wrong. The model here is a stand-in. It answers like a real one but always gives the same answer, so you can check every result.
By the end of this lesson
Note
I can send a request to a language model API and read every part of its answer.
Key words
- API: a way for one program to ask another for something.
- Message: one turn of a conversation, with a role and some text.
- Role: who a message is from:
system,userorassistant. - Token: a small piece of text a model reads or writes.
- Stream: an answer sent in pieces as it is made.
Starter: what you already know
Note
Answer in your head. 1. What does an HTTP status of 200 mean? 2. In JSON, what do the curly brackets {} hold? 3. If a website is busy, is it better to try again at once or after a pause? Answers: it worked; names and values; after a pause.
Before you start
Note
Press Start the lab on the bar at the top of this lesson and wait until it says it is running. There are two containers. model is the stand-in, answering at http://model:4000/v1, and the terminal's curl reaches it directly. tools has the lab's Python, and docker exec tools runs one command inside it. Each Run button below types its command into your lab's terminal, one after another in the same session. To watch them, or to type your own, press Open terminal on the bar.
A request is JSON, and so is the answer
The first command asks which models there are: one, called lab. The second sends one user message. In the answer, choices holds the reply, "The capital of France is Paris.", and finish_reason is stop: it ended by itself. usage counts 15 tokens in and 10 out.
curl -s -w "\n" http://model:4000/v1/modelsShown here, not run. Press Start lab to type it into your own lab’s terminal.
curl -s -w "\n" http://model:4000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"lab","messages":[{"role":"user","content":"What is the capital of France?"}]}'Shown here, not run. Press Start lab to type it into your own lab’s terminal.
The system message sets the rules, and nothing is remembered
The same question three times. A system message saying "Answer in capital letters." gives the reply in capitals, and "Only answer questions about cooking." gives a polite refusal. Then the user says "My name is Sam." The name is known only when the earlier messages are sent again. The model keeps nothing between requests: your program does.
docker exec tools python3 steps/roles.pyShown here, not run. Press Start lab to type it into your own lab’s terminal.
Tokens are what you are charged for
The stand-in splits "The lighthouse keeper counted ships." into 11 tokens. A whole story used 14 tokens in and 66 out. With max_tokens 12 the reply stops mid-word and finish_reason is length. At made-up prices of 50 pence a million tokens in and 200 out, one request costs 0.0139 pence, and 1,200 cost 16.68 pence.
docker exec tools python3 steps/tokens.pyShown here, not run. Press Start lab to type it into your own lab’s terminal.
Check
Tip
A reply ends with finish_reason length. What happened? Answer: it reached max_tokens and was cut short, so it may be unfinished. Ask for more tokens, or ask a shorter question.
Sampling settings change how a real model chooses
A real model picks each next token from likely ones. temperature near 0 nearly always takes the likeliest; higher takes more chances. seed repeats a choice. The script sends five settings and gets one reply: "the same every time: True". The stand-in ignores them, which a real model would not.
docker exec tools python3 steps/sampling.pyShown here, not run. Press Start lab to type it into your own lab’s terminal.
Structured output is JSON your program checks
Free text is hard for a program to use. Here the request offers a tool, record_person, with a schema, and forces the model to call it. The answer is tool_calls, with arguments as text: {"name": "Aiko", "age": 17, "city": "Leeds"}. The script parses it, checks every type, then uses the age as a number.
docker exec tools python3 steps/structured.pyShown here, not run. Press Start lab to type it into your own lab’s terminal.
Streams arrive in pieces, and errors need a plan
With "stream":true the answer comes as 14 events: an empty start, one per token, a finish, the usage, then [DONE]. The script then sends #busy, which the stand-in refuses with 429 two times in three. It waits 1 second, then 2, and the third try works. #broken always fails, so it gives up. A 400 is never retried.
curl -sN http://model:4000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"lab","stream":true,"stream_options":{"include_usage":true},"messages":[{"role":"user","content":"What is the capital of France?"}]}'Shown here, not run. Press Start lab to type it into your own lab’s terminal.
docker exec tools python3 steps/retry.pyShown here, not run. Press Start lab to type it into your own lab’s terminal.
A common mistake
Warning
Many people think the model remembers the conversation. It does not: each request stands alone, and the model knows only the messages in it. A chat app sends the whole conversation again every time, which is why long chats use more tokens and cost more.
Exit check
Tip
Answer without looking back. 1. Which role sets the rules for a conversation? 2. What does finish_reason length mean? 3. Why parse and check structured output? 4. Which errors are worth retrying? Answers: system; it hit max_tokens; it is text until your program checks it; busy ones like 429, after a growing pause.
For teachers
Tip
This lesson is a small example of a lesson worked in a lab: the steps and the Run buttons are the lesson, and the lab is each student's own copy. The lab is two services of one Python image: a stand-in model that answers in a real model API's format by fixed rules, and a Python with a client and short scripts. No real model is called, so it costs nothing and every output is the same. Key learning points: 1. A request is a list of messages, each with a role. 2. The model remembers only what each request sends. 3. Tokens limit a reply and set its cost. 4. Structured output must be parsed and checked. Common mistake: thinking the model remembers. Equipment: the lab and a browser. Tip: ask students to predict each output before they press Run, then say what a real model would do differently. Copy it into your own module and change it, or build one from scratch with the Lab menu in the lesson builder.