A chat call takes a list of messages
A chat call to a language model takes a list of messages, and each message has a role and some text. The model reads the whole list and writes the next message.
This matters because a model remembers nothing between calls. A chat that seems to remember is a program that sends the earlier messages again each time.
By the end of this lesson
Note
I can build the list of messages that a language model call takes.
Key words
| Word | What it means | Example |
|---|---|---|
| Message | one turn in a chat: a role and some text | |
| Role | who a message comes from | user |
| System message | the instruction that sets how the model behaves | |
| History | the earlier messages of a chat, sent again each call | |
| Request body | the JSON a program sends with a call |
The cells students fill in are left blank in this preview.
A message is a role and some text
Every message is an object with two keys. role says who is speaking and content holds the words.
There are three roles. The system message tells the model how to behave. A user message is what the person says. An assistant message is something the model said earlier.
The request body of one call
{
"model": "lab",
"max_tokens": 60,
"messages": [
{ "role": "system", "content": "You answer in one short sentence." },
{ "role": "user", "content": "What is a variable in programming?" }
]
}Send one call to the lab model
docker exec -w /home/node/project agent node -e '/* The request body: a model name, a limit on the reply, and a list of messages. */ const body = { model: "lab", max_tokens: 60, messages: [ { role: "system", content: "You answer in one short sentence." }, { role: "user", content: "What is a variable in programming?" } ] }; (async () => { try { const url = (process.env.AI_URL || "http://ai:4000") + "/v1/chat/completions"; const answer = await fetch(url, { method: "POST", body: JSON.stringify(body), headers: { "content-type": "application/json", authorization: "Bearer lab" } }); if (!answer.ok) throw new Error("HTTP " + answer.status); const data = await answer.json(); console.log("Model answer: " + data.choices[0].message.content); } catch (error) { console.log("No model answered (" + error.message + ")."); console.log("Recorded sample: A variable is a name that holds a value your program can change."); } })();'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The model remembers nothing between calls
Each call stands alone. The model sees only the messages in that one call.
To hold a conversation your program keeps a list. After each answer it adds the question and the reply. On the next call it sends the history again, with the new question at the end.
Write the helper file lab.js
docker exec -w /home/node/project agent node -e 'require("fs").writeFileSync("lab.js", ["// lab.js, part 1: the one place that sends a call to the model of the lab.", "const base = process.env.AI_URL || \"http://ai:4000\";", "", "async function send(body, timeout) {", " return fetch(base + \"/v1/chat/completions\", {", " method: \"POST\",", " headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },", " body: JSON.stringify({ model: \"lab\", ...body }),", " signal: AbortSignal.timeout(timeout || 30000),", " });", "}"].join("\n") + "\n")'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Add asking with a recorded sample
docker exec -w /home/node/project agent node -e 'require("fs").appendFileSync("lab.js", ["", "// Part 2: ask once. If the model cannot answer, give back the recorded sample instead.", "async function ask(body, sample) {", " try {", " const answer = await send(body);", " if (!answer.ok) throw new Error(\"HTTP \" + answer.status);", " const data = await answer.json();", " return { label: \"Model answer\", text: data.choices[0].message.content, data: data };", " } catch (error) {", " return { label: \"Recorded sample\", text: sample, note: error.message };", " }", "}", "", "module.exports = { send, ask };"].join("\n") + "\n")'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Ask with the history, then without it
docker exec -w /home/node/project agent node -e 'const lab = require("./lab.js"); const system = { role: "system", content: "Answer in one short sentence." }; const history = [ { role: "user", content: "My name is Sam." }, { role: "assistant", content: "Nice to meet you, Sam." } ]; const question = { role: "user", content: "What is my name?" }; (async () => { const withHistory = await lab.ask({ max_tokens: 30, messages: [system, ...history, question] }, "Your name is Sam."); console.log("With the history, " + withHistory.label + ": " + withHistory.text); const without = await lab.ask({ max_tokens: 30, messages: [system, question] }, "I do not know your name."); console.log("Without it, " + without.label + ": " + without.text); console.log("Messages sent each time: " + [system, ...history, question].length + " and " + [system, question].length); })();'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think the model remembers the last call. It does not. The second call without the history had no way to know the name. If you want the model to use something, put it in the messages you send.
Worked example: what each call carried
| Call | Messages sent | What the model can know |
|---|---|---|
| With the history | system, user, assistant, user (4) | The name Sam |
| Without it | system, user (2) | Nothing about Sam |
The cells students fill in are left blank in this preview.
The order of messages matters
Put the system message first. Then the turns alternate: user, assistant, user, and so on. The last message is from the user, because that is what the model answers.
Some model APIs refuse a list that breaks this order, and others give odd answers. Check the order in your own code before you send.
Going further: change the system message
Open the history Run and change the system message to "Answer in French". Press Run again.
Write down what changed and what did not. Then say in one sentence why the instruction had to be in the messages you sent.