A tool is a manual for a model
Designing a tool for an agent means writing its name, description, parameters and errors so that a model uses it correctly. Those words are the model's only manual.
A tool called do with five loose options gets guessed at. A tool called read_file with one path gets used well.
By the end of this lesson
Note
I can write a tool a model uses correctly, with a clear name, a stated limit and a helpful error.
Key words
| Word | What it means |
|---|---|
| Tool | a function the agent may ask the harness to run |
| Description | the words that tell the model when to use a tool |
| Narrow tool | a tool that does one job |
| Parameter | a named input of a tool |
| Ambiguity | two tools that look right for one task |
| Tool error | the text a tool returns when it fails |
The cells students fill in are left blank in this preview.
Narrow tools beat broad ones
A narrow tool does one job and takes few parameters. A broad tool, such as run_anything(options), forces the model to invent a mini language, and it often gets it wrong.
Give each job its own tool: read_file, write_file, list_files. Keep to four parameters or fewer. A tool that needs more is usually two tools.
Names and descriptions say when not to
A name is a verb and a noun: search_text, not helper. A description says what the tool does, and also when not to use it.
That second sentence is what separates two tools that look alike. Without it, the model picks by luck between read_file and search_text, and you find out in a trace.
Faults to check for
| Fault | Looks like | Fix |
|---|---|---|
| Vague name | `do`, `helper` | A verb and a noun |
| Short description | `Does things` | Say what it reads or changes |
| No limit | No "do not use" | Add what it is not for |
| Too many parameters | Five or more | Split it into tools |
The cells students fill in are left blank in this preview.
Write errors for the model
A tool error is the next input the model reads. So write it for the model: what went wrong, then what to do next.
Error: no file at src/app.pyy. Call list_files, then try again. tells the model how to recover. A bare ENOENT, the system code for a missing file, does not. For a refusal, say not to retry, or the model will try again and again.
A common mistake
Warning
Many people write a tool's words for themselves and keep them short. The model has nothing else to go on. Write the description as if for a colleague who has never seen the code.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the tool checker, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const VAGUE = [\"do\", \"run\", \"handle\", \"tool\", \"helper\", \"process\", \"util\"];' 'const tools = [' ' { name: \"read_file\", description: \"Read the text of one file by path. Do not use it for folders.\", params: [\"path\"] },' ' { name: \"search_text\", description: \"Find lines that hold a word in every file of the project\", params: [\"word\"] },' ' { name: \"do\", description: \"Does things\", params: [\"a\", \"b\", \"c\", \"d\", \"e\"] },' '];' > tools2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the tool checker, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function faults(tool) {' ' const found = [];' ' if (VAGUE.includes(tool.name)) found.push(\"vague-name\");' ' if (tool.description.split(\" \").length < 8) found.push(\"short-description\");' ' if (tool.description.toLowerCase().includes(\"do not use\") === false) found.push(\"no-limit\");' ' if (tool.params.length > 4) found.push(\"too-many-params\");' ' return found;' '}' >> tools2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the tool checker, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function errorText(e) {' ' if (e.code === \"not_found\") return \"Error: no file at \" + e.path + \". Call list_files, then try again.\";' ' if (e.code === \"too_big\") return \"Error: \" + e.path + \" has \" + e.size + \" lines and the limit is \" + e.limit + \". Ask for a range of lines.\";' ' return \"Error: \" + e.code + \". Do not retry.\";' '}' >> tools2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the tool checker, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' for (const t of tools) console.log(t.name + \": \" + (faults(t).join(\", \") || \"no faults\"));' ' console.log(errorText({ code: \"not_found\", path: \"src/app.pyy\" }));' ' console.log(errorText({ code: \"too_big\", path: \"data.csv\", size: 9000, limit: 500 }));' ' const list = tools.slice(0, 2).map(function (t) { return t.name + \": \" + t.description; }).join(\" | \");' ' const task = \"Show me the first lines of src/app.py. Answer with one tool name only.\";' ' const reply = await ask([{ role: \"user\", content: \"Tools: \" + list + \" | \" + task }], \"read_file\", 10);' ' console.log(\"model chose: \" + reply.text);' '}' 'main();' >> tools2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the tool checker
docker exec -w /home/node/project agent node tools2.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the one live call shows
The lab's model is reached as plain text, with no tool-calling feature. So the Run pastes the tool descriptions into the prompt and asks for a tool name. That is enough to show that the words are the manual.
The checks and the two errors need no model at all.
One possible output
| Line | What it shows |
|---|---|
| read_file: no faults | A narrow tool with a limit |
| search_text: no-limit | It never says when not to use it |
| do: vague-name, short-description, no-limit, too-many-params | Four faults in one tool |
| Error: no file at src/app.pyy. Call list_files, then try again. | An error with a next step |
| model chose: ... | Probably read_file. The words differ each run |
The cells students fill in are left blank in this preview.