Put the five parts together
A working chatbot keeps state, trims it, follows a system prompt, streams its answer and shows its sources, all in one loop. It matters because each part works alone, and the bugs appear where they meet.
A library chatbot answers three questions in a row. It must remember the first, stay inside its budget and cite a source each time.
By the end of this lesson
Note
I can put state, trimming, a prompt, streaming and sources into one chatbot.
Key words
| Word | What it means |
|---|---|
| Chat loop | the steps repeated for every user message |
| Turn | one user message and the reply to it |
| History | every message of the chat so far |
| Budget | the most tokens you choose to send |
| Fallback | what the bot does when no model answers |
The cells students fill in are left blank in this preview.
One turn uses all five parts
Every turn runs the same steps. Load the history. Add the user's message. Trim what you will send. Call the model. Check the reply. Save the history.
Write these as one function, turn. Then the chat is that function called again and again.
Worked example: the steps of one turn
| Step | From lesson | What happens |
|---|---|---|
| Load and add | Conversation state | The user's message joins the session's list |
| Trim | Trimming history | Old messages are left out of this call |
| Prompt and sources | System prompts, Citations | The system message holds the rules and numbered sources |
| Call and check | Citations | The reply is checked for markers that name no source |
| Stream and save | A streaming web UI | Pieces go to the page, the history is saved |
The cells students fill in are left blank in this preview.
What is kept is not what is sent
The history keeps every message. The call sends only what fits the budget. So trimming never changes the record. It changes one call.
This also means you can change the budget later, and a person can still read the whole chat.
Plan for the model not answering
Sometimes no model answers: AI is off, the day's credit is used or the network drops. A fallback is a clear message that says so, so the chat does not hang or crash.
In this lab every Run prints a labelled recorded sample in that case. A real product would show its own notice.
A common mistake
Warning
Many people think a chatbot is finished when the demo works. A demo uses friendly questions. Try awkward ones, a very long chat and no model at all. Count the tokens one chat costs before other people use it.
Write the model helper
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 200 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' return (await res.json()).choices[0].message.content;' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' const list = [].concat(sample);' ' const turn = messages.filter(function (m) { return m.role === \"assistant\"; }).length;' ' return list[Math.min(turn, list.length - 1)];' ' }' '}' 'module.exports = { ask };' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the chatbot, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const fs = require(\"fs\");' 'const { ask } = require(\"./model\");' 'const BUDGET = 150; // tokens for the system message and the history together' 'const SOURCES = [' ' \"Members may borrow up to six books for three weeks.\",' ' \"A late book costs ten pence a day.\",' ' \"The library opens at nine and closes at five on weekdays.\",' '];' > chatbot6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the chatbot, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function tokens(m) { return Math.ceil(m.content.length / 4) + 4; }' 'function total(list) {' ' return list.reduce(function (sum, m) { return sum + tokens(m); }, 0);' '}' 'function systemMessage() {' ' const numbered = SOURCES.map(function (t, at) { return \"[\" + (at + 1) + \"] \" + t; }).join(\" \");' ' const rule = \"You are the library helper. Answer in at most two sentences, using only these sources. Write the source number in square brackets after each fact. If they do not say, reply: I do not know. \";' ' return { role: \"system\", content: rule + numbered };' '}' >> chatbot6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the chatbot, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function trim(history, system) {' ' const kept = history.slice();' ' while (total([system].concat(kept)) > BUDGET && kept.length > 1) kept.shift();' ' return kept;' '}' 'function badMarkers(reply) {' ' const bad = [];' ' const re = /\[(\d+)\]/g;' ' let m = re.exec(reply);' ' while (m) {' ' if (Number(m[1]) < 1 || Number(m[1]) > SOURCES.length) bad.push(m[1]);' ' m = re.exec(reply);' ' }' ' return bad;' '}' >> chatbot6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the chatbot, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function turn(state, text, sample) {' ' state.history.push({ role: \"user\", content: text });' ' const system = systemMessage();' ' const sent = trim(state.history, system);' ' const held = state.history.length;' ' const reply = await ask([system].concat(sent), sample, 70);' ' state.history.push({ role: \"assistant\", content: reply });' ' fs.writeFileSync(\"chat.json\", JSON.stringify(state, null, 1));' ' console.log(\"You: \" + text);' ' console.log(\"Bot: \" + reply);' ' console.log(\" sent \" + sent.length + \" of \" + held + \" messages, bad markers: \" + badMarkers(reply).length);' '}' >> chatbot6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the chatbot, part 5
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' const state = { history: [] };' ' // Recorded samples, used only when no model answers.' ' await turn(state, \"How many books can I borrow?\", \"You can borrow six books for three weeks [1].\");' ' await turn(state, \"And how much is a late fee?\", \"A late book costs ten pence a day [2].\");' ' await turn(state, \"When do you close on a Friday?\", \"The library closes at five on weekdays [3].\");' ' console.log(\"Saved \" + state.history.length + \" messages in chat.json\");' '}' 'main();' >> chatbot6.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Hold a three turn chat
docker exec -w /home/node/project agent node chatbot6.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
One possible output
| Line | What it may show |
|---|---|
| You and Bot | Three questions, each with a short answer that may cite a source |
| sent n of m | sent 1 of 1, then 3 of 3, then 5 of 5 or fewer. The recorded replies give 4 of 5 |
| bad markers | Usually 0. A number above 0 means the reply cited a source that does not exist |
| Saved | Saved 6 messages in chat.json |
The cells students fill in are left blank in this preview.