The model asks for a file
Limiting a model's tools means that your code, not the model, decides what each tool may do. It matters because a model's request is text, and text can be wrong, tricked or invented.
A notes assistant has a tool that reads a note by name. A reply asks for the note called "../private". The tool would happily try. A policy stops it before it starts.
By the end of this lesson
Note
I can check each tool request against an allow-list policy before anything runs.
Key words
| Word | What it means |
|---|---|
| Allow-list | the only tools and values that are permitted |
| Policy | data that says what each tool may be given |
| Argument check | a test of a tool's inputs before it runs |
| Least privilege | giving a tool only the power it needs |
| Timeout | a limit on how long a call may take |
The cells students fill in are left blank in this preview.
An allow-list names what is permitted
A block-list names bad things and lets the rest through. It always misses one. An allow-list names the good things and refuses the rest.
Write a policy as data: each tool, each argument, its type and its limits. One function reads the policy and judges every request. The model never runs anything itself.
Worked example: a policy and four requests
| Request | Policy says | Decision |
|---|---|---|
| read_note, name menu | name must be todo, menu or plan | Allowed |
| read_note, name ../private | name must be todo, menu or plan | Refused: not allowed |
| run_shell, any text | No such tool | Refused: unknown tool |
| add_numbers, b 99999 | b is at most 1000 | Refused: too big |
The cells students fill in are left blank in this preview.
Never build a command from model text
Some tools are tempting: run a command, evaluate an expression, write a file. Do not build any of them from text a model wrote. A refusal is safer than a clever filter.
If a feature needs real code to run, run it in a separate sandbox, with no secrets and a time limit. This lab has none, so the lesson only refuses.
Write the model helper
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 200 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' return (await res.json()).choices[0].message.content;' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' const list = [].concat(sample);' ' const turn = messages.filter(function (m) { return m.role === \"assistant\"; }).length;' ' return list[Math.min(turn, list.length - 1)];' ' }' '}' 'module.exports = { ask };' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the policy script, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' '' '// Made-up notes, and a policy: each tool, its arguments and their limits.' 'const notes = { todo: \"buy flour\", menu: \"bread, cake\", plan: \"open at 8\" };' 'const policy = {' ' read_note: { name: { type: \"string\", allowed: Object.keys(notes) } },' ' add_numbers: { a: { type: \"number\", max: 1000 }, b: { type: \"number\", max: 1000 } },' '};' > policy.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the policy script, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function check(request) {' ' const rule = policy[request.tool];' ' if (rule === undefined) return \"refused: unknown tool\";' ' const args = request.args || {};' ' for (const key of Object.keys(args)) {' ' if (rule[key] === undefined) return \"refused: unexpected argument \" + key;' ' }' ' for (const key of Object.keys(rule)) {' ' const spec = rule[key];' ' const value = args[key];' ' if (typeof value === spec.type === false) return \"refused: bad \" + key;' ' if (spec.allowed && spec.allowed.includes(value) === false) return \"refused: \" + key + \" not allowed\";' ' if (typeof spec.max === \"number\" && value > spec.max) return \"refused: \" + key + \" too big\";' ' }' ' return \"ok\";' '}' >> policy.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the policy script, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function run(request) {' ' if (check(request) === \"ok\") {' ' return request.tool === \"read_note\" ? notes[request.args.name] : request.args.a + request.args.b;' ' }' ' return check(request);' '}' 'const scripted = [' ' { tool: \"read_note\", args: { name: \"menu\" } },' ' { tool: \"read_note\", args: { name: \"../private\" } },' ' { tool: \"run_shell\", args: { text: \"list the folder\" } },' ' { tool: \"add_numbers\", args: { a: 5, b: 99999 } },' ' { tool: \"add_numbers\", args: { a: 2, b: 3, c: 4 } },' '];' 'for (const request of scripted) console.log(request.tool + \" -> \" + run(request));' >> policy.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the policy script, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function pullJson(text) {' ' const start = text.indexOf(\"{\");' ' const end = text.lastIndexOf(\"}\");' ' try { return JSON.parse(text.slice(start, end + 1)); } catch (error) { return null; }' '}' 'async function main() {' ' const ask2 = [{ role: \"user\", content: \"Reply with only a JSON object with the keys tool and args, to read the note called plan. Tools: read_note(name), add_numbers(a, b).\" }];' ' const sample = JSON.stringify({ tool: \"read_note\", args: { name: \"plan\" } });' ' const request = pullJson(await ask(ask2, sample, 80));' ' console.log(\"Model request: \" + JSON.stringify(request));' ' console.log(\"Result: \" + (request === null ? \"refused: not JSON\" : run(request)));' '}' 'main();' >> policy.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the policy script
docker exec -w /home/node/project agent node policy.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
One possible output
| Line | What it shows |
|---|---|
| read_note -> bread, cake | The first request passed every check |
| read_note -> refused: name not allowed | The path trick failed the allow-list |
| run_shell -> refused: unknown tool | The tool is not in the policy |
| Model request, then Result | Differs each run. A tool request from the model goes through the same check |
The cells students fill in are left blank in this preview.
Limit the time and the power
A timeout stops a slow tool from holding a request for ever. The helper you wrote already gives up on the model after 30 seconds.
Apply least privilege too. A tool that reads notes should not be able to write them, and a tool that writes should only reach its own folder.
A common mistake
Warning
Many people think that if the model was told the rules, the tool does not need to check. The model may be wrong or tricked, or it may invent a tool. Always check in the code that runs the tool, because that code is the only part that does the same thing every time.