A chatbot is a loop around one call
A command-line assistant is a program that reads a line you type and sends it to a language model. It prints the reply and repeats until you quit. It is the smallest useful program built on a model.
It matters because it uses everything so far: messages, a history, a stream and a failed call that must not break the chat.
By the end of this lesson
Note
I can put model calls together into a program that a person can use.
Key words
| Word | What it means | Example |
|---|---|---|
| Loop | code that repeats until it is told to stop | |
| Command | a typed line that controls the program, not the model | /quit |
| History | the earlier messages that each call sends again | |
| Trim | to drop the oldest messages so the history stays short | |
| Clean exit | leaving on purpose, with no error and nothing lost |
The cells students fill in are left blank in this preview.
The program is a loop with a list
Each turn does the same steps. Read a line. Add it to the history as a user message. Send the messages. Print the reply. Add the reply to the history as an assistant message.
The list is the program's whole memory. If it is wrong, every later answer is wrong too.
Write part 1: the history and its limit
docker exec -w /home/node/project agent node -e 'require("fs").writeFileSync("assistant.js", ["// assistant.js, part 1: the history, and a limit on how long it grows.", "const lab = require(\"./lab.js\");", "const readline = require(\"readline\");", "const system = { role: \"system\", content: \"You are a helpful assistant. Answer in one or two short sentences.\" };", "const history = [];", "const LIMIT = 6;", "", "// Keep only the newest messages, and never start with an assistant message.", "function trim() {", " while (history.length > LIMIT) history.shift();", " while (history.length > 0 && history[0].role === \"assistant\") history.shift();", "}"].join("\n") + "\n")'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Trim the history so it stays short
Every call sends the whole history, so a long chat costs more with each turn and may pass the size the model can read. Trim it: keep only the newest messages.
After a cut the list must not start with an assistant message. A chat should start with the user, as you saw in the first lesson.
Add part 2: stream the reply
docker exec -w /home/node/project agent node -e 'require("fs").appendFileSync("assistant.js", ["", "// Part 2: send the history and print the stream as it arrives.", "const { piecesOf } = require(\"./stream.js\");", "async function streamReply() {", " let reply = \"\";", " for await (const piece of piecesOf({ max_tokens: 120, messages: [system, ...history] })) {", " reply += piece;", " process.stdout.write(piece);", " }", " if (reply === \"\") throw new Error(\"no words came back\");", " return reply;", "}"].join("\n") + "\n")'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Add part 3: keep the history in step
docker exec -w /home/node/project agent node -e 'require("fs").appendFileSync("assistant.js", ["", "// Part 3: keep the history in step. A failed call takes the question out again.", "async function answer(text) {", " history.push({ role: \"user\", content: text });", " try {", " const reply = await streamReply();", " process.stdout.write(\"\\n\");", " history.push({ role: \"assistant\", content: reply });", " trim();", " } catch (error) {", " history.pop();", " console.log(\"\\nRecorded sample: I cannot reach the model now, so this is a recorded answer. (\" + error.message + \")\");", " }", "}"].join("\n") + "\n")'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people leave the question in the history when a call fails. Then the next turn sends two user messages in a row, and the order breaks. When a call fails, take the question out again, say so, and let the person try again.
Commands and a clean exit
Some lines are for the program, not the model. A line that starts with / is a command: /quit leaves, /clear empties the history and /history counts it.
Check for commands before you call the model. Then a person can always leave, whatever the model is doing.
Add part 4: the loop and the commands
docker exec -w /home/node/project agent node -e 'require("fs").appendFileSync("assistant.js", ["", "// Part 4: the loop. A line that starts with / is a command for the program, not a question.", "const lines = readline.createInterface({ input: process.stdin });", "const prompt = () => process.stdout.write(\"you> \");", "(async () => {", " console.log(\"Assistant ready. Type /quit to leave, /clear to forget and /history to count.\");", " prompt();", " for await (const line of lines) {", " const text = line.trim();", " if (!process.stdin.isTTY) console.log(text);", " if (text === \"/quit\") break;", " if (text === \"/clear\") { history.length = 0; console.log(\"History cleared.\"); }", " else if (text === \"/history\") console.log(history.length + \" messages kept.\");", " else if (text !== \"\") { process.stdout.write(\"assistant> \"); await answer(text); }", " prompt();", " }", " console.log(\"Bye.\");", " process.exit(0);", "})();"].join("\n") + "\n")'Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Chat with a typed script
printf "My name is Sam.\nWhat is my name?\n/history\n/clear\n/history\n/quit\n" | docker exec -i -w /home/node/project agent node assistant.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Worked example: what the script did
| Typed | What the program did |
|---|---|
| My name is Sam. | Sent 2 messages (system and question), printed a reply, kept 2 |
| What is my name? | Sent 4 messages, printed a reply, kept 4 |
| /history | Printed 4 messages kept |
| /clear | Emptied the history |
| /quit | Printed Bye and left |
The cells students fill in are left blank in this preview.
Chat with it yourself
The script above was typed for you. To chat by hand, open a shell in the lab and type this line: docker exec -it -w /home/node/project agent node assistant.js. Then type questions and end with /quit.
It waits for you to type, so it is not a button. Do not type a password or anything personal: what you type is sent to a model.
Going further: change the assistant
Change the system message in part 1 to give the assistant a personality, and change LIMIT to 4. Chat for six turns.
Write down what the assistant forgot and when. Say what it cost you to keep a long history, and what you would do about it.