Tools fail, and the model can help
When a tool fails, your program should pass the error back to the model as the tool's result. It matters because a crash stops the whole assistant, while an error the model can read lets it recover.
A tool can fail in several ways. The item may not exist, the service may be slow, or the model may send a bad value.
By the end of this lesson
Note
I can pass a tool's error back as a result so the model can recover.
Key words
| Word | What it means |
|---|---|
| Exception | an error a function raises when it cannot finish |
| Error result | a tool message that says what went wrong |
| Retry | to try the same call again |
| Timeout | a call that took too long and was given up |
| Give up | to stop trying and report the failure |
The cells students fill in are left blank in this preview.
An error is a result to read
Wrap the call in a try. When it raises, do not stop. Put the message in a tool message, such as Error: no such item: kiwi, and ask the model again.
The model sees the words. It may say sorry to the user or try another item. The loop still has its step limit, so it cannot go on for ever.
Worked example: what each failure becomes
| What went wrong | Retry? | Message to the model |
|---|---|---|
| The item is not in the shop | No | Error: not found: kiwi |
| The value is not allowed | No | Error: negative number |
| The arguments do not fit | No | Error: bad arguments |
| The service timed out | Yes, a few times | Error: timed out, after the last try |
The cells students fill in are left blank in this preview.
Save a helper that calls the model
docker exec -w /home/node/project agent node -e "const ask=async(m,t,c)=>{const r=await fetch((process.env.AI_URL||'http://ai:4000')+'/v1/chat/completions',{method:'POST',headers:{'content-type':'application/json',authorization:'Bearer lab'},body:JSON.stringify({model:'lab',messages:m,tools:t,tool_choice:c,max_tokens:120})});const j=await r.json();if(r.ok)return j.choices[0].message;throw Error(r.status+' '+j.error.message)};require('fs').writeFileSync('ai.cjs','exports.ask='+ask+';');console.log('Saved ai.cjs: ask(messages, tools) returns the model message.')"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Wrap a failing tool and a flaky one
docker exec -w /home/node/project agent node -e "let n=0;const flaky=()=>{n++;if(n<3)throw Error('timeout');return 'ok'};const price=a=>{if((a.item=='apple')===false)throw Error('no such item: '+a.item);return 40};const call=(f,a,tries)=>{for(let i=1;;i++){try{return{ok:true,result:String(f(a)),attempts:i}}catch(e){if(e.message=='timeout'&&i<tries)continue;return{ok:false,error:e.message,attempts:i}}}};console.log(JSON.stringify(call(price,{item:'apple'},3)));console.log(JSON.stringify(call(price,{item:'kiwi'},3)));console.log(JSON.stringify(call(flaky,{},3)))"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Retry only what might work next time
A timeout may pass, so trying again can help. A missing item will still be missing, so a retry only wastes time.
Set a number of tries, count each call, and give up with a clear message when they run out. Report how many attempts you made.
Pass an error back to a model
docker exec -w /home/node/project agent node -e "const{ask}=require('./ai.cjs');const P={apple:40,pear:45};const T=[{type:'function',function:{name:'get_price',description:'Get a price.',parameters:{type:'object',properties:{item:{type:'string'}}}}}];const go=async(m,f)=>{try{return await ask(m,T)}catch(e){console.log('No model ('+e.message+'). Recorded sample:');return f}};(async()=>{const m=[{role:'user',content:'How much is a kiwi?'}];const a=await go(m,{role:'assistant',tool_calls:[{id:'c1',function:{name:'get_price',arguments:JSON.stringify({item:'kiwi'})}}]});m.push(a);const c=a.tool_calls?.[0];if(c==null)return console.log('Answer: '+a.content);const i=JSON.parse(c.function.arguments).item;const r=i in P?String(P[i]):'Error: no such item: '+i;console.log('Tool result: '+r);m.push({role:'tool',tool_call_id:c.id,content:r});const b=await go(m,{role:'assistant',content:'Sorry, no kiwi.'});console.log('Next: '+(b.content||JSON.stringify(b.tool_calls[0].function)))})()"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A common mistake
Warning
Many people think a failed tool means the whole conversation has failed. It does not. An error is only one more result. Send it back as text, keep the step limit, and let the model decide what to do.
Going further: change the tries
In the wrapping Run, find the last call, call(flaky,{},3), and change its 3 to 2. Run it again and look at the last line.
Write down what changes, and say why a limit on tries matters.