The loop behind every tool-using model
The tool calling loop is how a model uses the result of a tool. It matters because every assistant that uses tools runs this loop, and the loop must always have a way to stop.
A tool call is only half a conversation. The model asks for a call, your program runs it, and the model must then see the result before it can answer. So the exchange is a loop.
By the end of this lesson
Note
I can run the loop of ask, call, answer and ask again.
Key words
| Word | What it means |
|---|---|
| Loop | steps your program repeats until a stop condition |
| Message history | the list of messages sent with every ask |
| Tool result | the message that carries a call's answer back to the model |
| Step | one ask and the reply to it |
| Step limit | the most steps a loop may take |
The cells students fill in are left blank in this preview.
The loop asks again after each call
Send the messages and the tools to the model. If the reply has tool_calls, run each call, add the results to the messages and ask again. If the reply is plain words, that is the answer and the loop ends.
The model never keeps anything between asks. Each ask carries the whole history.
Worked example: the history after each step
| Step | What happens | Messages sent next |
|---|---|---|
| Start | The user asks about a pear | user |
| 1 | The model asks for get_price | user, assistant |
| 1 | Your program adds the result | user, assistant, tool |
| 2 | The model answers in words | user, assistant, tool, assistant |
The cells students fill in are left blank in this preview.
Save a helper that calls the model
docker exec -w /home/node/project agent node -e "const ask=async(m,t,c)=>{const r=await fetch((process.env.AI_URL||'http://ai:4000')+'/v1/chat/completions',{method:'POST',headers:{'content-type':'application/json',authorization:'Bearer lab'},body:JSON.stringify({model:'lab',messages:m,tools:t,tool_choice:c,max_tokens:120})});const j=await r.json();if(r.ok)return j.choices[0].message;throw Error(r.status+' '+j.error.message)};require('fs').writeFileSync('ai.cjs','exports.ask='+ask+';');console.log('Saved ai.cjs: ask(messages, tools) returns the model message.')"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A tool result carries the call id
After a call you add a message with the role tool. It holds the tool_call_id of the request it answers and the result as text.
The model uses the id to match each result to its request. Without it, a reply with two calls could not be answered.
Run the loop for up to three steps
docker exec -w /home/node/project agent node -e "const{ask}=require('./ai.cjs');const P={apple:40,pear:45};const T=[{type:'function',function:{name:'get_price',description:'Get a shop item price.',parameters:{type:'object',properties:{item:{type:'string'}}}}}];let rec=0;const fake=m=>m.at(-1).role=='tool'?{role:'assistant',content:'A pear costs 45p.'}:{role:'assistant',tool_calls:[{id:'c1',function:{name:'get_price',arguments:JSON.stringify({item:'pear'})}}]};(async()=>{const m=[{role:'user',content:'How much is a pear?'}];for(let s=1;s<4;s++){let a;try{if(rec)throw 0;a=await ask(m,T)}catch(e){rec||console.log('No model ('+e.message+'). Recorded sample:');rec=1;a=fake(m)}m.push(a);if(a.tool_calls==null){console.log('Step '+s+': answer: '+a.content);return}for(const c of a.tool_calls){const v=P[JSON.parse(c.function.arguments).item];console.log('Step '+s+': '+c.function.name+' -> '+v);m.push({role:'tool',tool_call_id:c.id,content:String(v)})}}console.log('Limit reached')})()"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
A step limit ends a runaway loop
A model may ask for a call again and again. Each step costs time and credit, so you count the steps and stop at a limit. Three is a sensible start.
When the limit comes first, your program says so. It does not pretend it has an answer.
A common mistake
Warning
Many people think the model remembers earlier asks. It does not. It only sees the messages you send. If you leave out the tool result, the model cannot know what the call returned, so keep every message.
Going further: change the question
Change the question in the Run to How much are an apple and a pear?. Run it three times and count the steps each time.
Then change s<4 to s<2. Write down what the program prints when the limit comes first.