What happens when a model has a toolbox
Giving a model several tools means offering it a small toolbox and sending each request it makes to the right function. It matters because a real assistant needs more than one tool, and the model can ask for a tool that does not exist.
Picture a price lookup, a unit converter and a place to save notes. The model picks one for each request. Your program must route it, or refuse it.
By the end of this lesson
Note
I can offer more than one tool and route each request to the right function.
Key words
| Word | What it means |
|---|---|
| Router | code that sends a request to the right function |
| Tool table | a table that maps each tool name to its function |
| Unknown tool | a name that is not in your tool table |
| Required argument | a value the tool cannot run without |
| Validation | checking a request before you act on it |
The cells students fill in are left blank in this preview.
A tool table maps names to functions
Keep your tools in one table. The key is the name the model uses. The value is your function.
To route a request, look the name up in the table and call what you find. Adding a tool is then one new line, and the table is the only list of what the model may use.
Worked example: a toolbox of three tools
| Name | What it does | Required argument |
|---|---|---|
| get_price | Looks up a shop price | item |
| km_to_miles | Converts kilometres to miles | km |
| add_note | Saves a short note | text |
The cells students fill in are left blank in this preview.
Save a helper that calls the model
docker exec -w /home/node/project agent node -e "const ask=async(m,t,c)=>{const r=await fetch((process.env.AI_URL||'http://ai:4000')+'/v1/chat/completions',{method:'POST',headers:{'content-type':'application/json',authorization:'Bearer lab'},body:JSON.stringify({model:'lab',messages:m,tools:t,tool_choice:c,max_tokens:120})});const j=await r.json();if(r.ok)return j.choices[0].message;throw Error(r.status+' '+j.error.message)};require('fs').writeFileSync('ai.cjs','exports.ask='+ask+';');console.log('Saved ai.cjs: ask(messages, tools) returns the model message.')"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Refuse a request you cannot serve
A model can ask for a name that is not in your table. It can send arguments that do not parse. It can leave a required one out. Check all three before you call anything.
A refusal is a message back to the model, such as error: unknown tool delete_all. It is never a crash, and nothing is run.
Route four requests, two of them bad
docker exec -w /home/node/project agent node -e "const F={get_price:a=>({apple:40,pear:45})[a.item],km_to_miles:a=>a.km*0.62,add_note:a=>'saved'};const R={get_price:['item'],km_to_miles:['km'],add_note:['text']};const route=c=>{if(Object.hasOwn(F,c.name)===false)return 'error: unknown tool '+c.name;let a;try{a=JSON.parse(c.arguments)}catch(e){return 'error: bad arguments'}for(const k of R[c.name])if((k in a)===false)return 'error: missing '+k;return String(F[c.name](a))};for(const c of [{name:'get_price',arguments:JSON.stringify({item:'pear'})},{name:'delete_all',arguments:'{}'},{name:'get_price',arguments:'{item:'},{name:'km_to_miles',arguments:'{}'}])console.log(c.name+' -> '+route(c))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Offer three tools to a model
docker exec -w /home/node/project agent node -e "const{ask}=require('./ai.cjs');const t=(n,d,k)=>({type:'function',function:{name:n,description:d,parameters:{type:'object',properties:{[k]:{type:k=='km'?'number':'string'}}}}});const T=[t('get_price','Get a shop item price in pence.','item'),t('km_to_miles','Convert kilometres to miles.','km'),t('add_note','Save a short note.','text')];const F={get_price:a=>({apple:40,pear:45})[a.item],km_to_miles:a=>Math.round(a.km*621.371)/1000,add_note:a=>'saved: '+a.text};(async()=>{let c;try{const m=await ask([{role:'user',content:'How many miles is 10 km?'}],T,'required');c=m.tool_calls[0]}catch(e){console.log('No model ('+e.message+'). Recorded sample:');c={function:{name:'km_to_miles',arguments:JSON.stringify({km:10})}}}const f=c.function;console.log('Request: '+f.name+' '+f.arguments);console.log('Result: '+(Object.hasOwn(F,f.name)?F[f.name](JSON.parse(f.arguments)):'unknown tool'))})()"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think the model only asks for tools you offered. It can ask for any name, and it can send arguments that do not fit. Always check the name against your table and the arguments against what the tool needs.
Going further: add a fourth tool
In the router Run, add a tool called shout that returns its text argument in capital letters. Add it to F and to R, then send a request for it.
Write down how many places you had to change.