Put the team together
A research team of agents is an orchestrator, a few workers, a check on every result and a rule for when to stop. It matters because every part so far has a cost, and a team without a limit has no end.
This lesson ties the module into one loop with a token budget.
By the end of this lesson
Note
I can run an orchestrator and workers with a budget, a check and a stop rule.
Key words
| Word | What it means |
|---|---|
| Budget | the most tokens a whole run may spend |
| Stop rule | a test that ends the run before the briefs run out |
| Accepted | a result that passed every check |
| Rejected | a result left out, with its reasons |
| Spent | the tokens used so far, good results and bad |
| Enough | the number of accepted results that ends the run |
The cells students fill in are left blank in this preview.
One loop, three limits
The orchestrator takes the briefs in order. Before each one it asks two questions. Do I have enough already? Could this brief push me over the budget?
Then it asks the worker and checks the result. Every reply is paid for, so spent counts good and bad alike.
Worked example: one pass of the loop
| Step | Question | If yes |
|---|---|---|
| 1 | Are enough results accepted? | Stop, reason enough |
| 2 | Spent plus this brief's limit over the budget? | Stop, reason budget |
| 3 | Ask the worker, add its tokens to spent | Always, good or bad |
| 4 | Does the result pass the check? | Accept it, else reject with reasons |
The cells students fill in are left blank in this preview.
Why a limit goes before the call
Checking the budget after a call is too late: the tokens are gone. So the loop reserves the worst case. It uses the brief's own limit, and stops if that would not fit.
A worker can still go over its own limit, as the third one does in the Run below. So a real system also caps the size of each reply. A team always has a rule that ends it.
A common mistake
Warning
Many people count only the good results against the budget. A worker that fails still reads and writes tokens. Count everything the run asked for, or the bill will not match the budget.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the research team, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const BRIEFS = [' ' { id: \"b1\", question: \"How many visitors want Sunday opening?\", sources: [\"s1\"], max: 900 },' ' { id: \"b2\", question: \"What would a Sunday cost in staff time?\", sources: [\"s2\"], max: 900 },' ' { id: \"b3\", question: \"How busy is a weekend day?\", sources: [\"s3\"], max: 900 },' '];' '// The replies of three scripted workers, recorded.' 'const RECORDED = {' ' b1: { answer: \"212 of 400 visitors want it\", sources: [\"s1\"], tokens: 700 },' ' b2: { answer: \"108 pounds each Sunday\", sources: [\"s2\"], tokens: 850 },' ' b3: { answer: \"310 visits on a Saturday\", sources: [\"s3\"], tokens: 1200 },' '};' > team.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the research team, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function problems(b, r) {' ' const out = [];' ' if (r.answer.trim().length === 0) out.push(\"missing answer\");' ' r.sources.forEach(function (s) { if (b.sources.includes(s) === false) out.push(\"unknown source \" + s); });' ' if (r.tokens > b.max) out.push(\"over its limit\");' ' return out;' '}' >> team.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the research team, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function team(budget, enough) {' ' const out = { accepted: [], rejected: [], spent: 0, stopped: \"done\" };' ' for (const b of BRIEFS) {' ' if (out.accepted.length >= enough) { out.stopped = \"enough\"; break; }' ' if (out.spent + b.max > budget) { out.stopped = \"budget\"; break; }' ' const r = RECORDED[b.id];' ' out.spent += r.tokens;' ' const p = problems(b, r);' ' if (p.length > 0) out.rejected.push(b.id + \" (\" + p.join(\", \") + \")\");' ' else out.accepted.push(r.answer);' ' }' ' return out;' '}' >> team.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the research team, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' let first = null;' ' for (const run of [[2500, 3], [1500, 3], [4000, 2]]) {' ' const o = team(run[0], run[1]);' ' first = first || o;' ' console.log(\"budget \" + run[0] + \", enough \" + run[1] + \": accepted \" + o.accepted.length + \", rejected \" + o.rejected.length + \", spent \" + o.spent + \", stopped \" + o.stopped);' ' if (o.rejected.length > 0) console.log(\" rejected: \" + o.rejected.join(\"; \"));' ' }' ' const q = [{ role: \"user\", content: \"Write one sentence from these findings: \" + first.accepted.join(\"; \") }];' ' const r = await ask(q, \"Visitors want Sunday opening and it costs 108 pounds a Sunday.\", 40);' ' console.log(\"Summary: \" + r.text.trim());' '}' 'main();' >> team.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the research team
docker exec -w /home/node/project agent node team.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The workers are recorded
The three workers are recorded replies, with their token counts, so every student sees the same totals. The one live call words a summary of what was accepted.
A real team would send each brief to a model, and the loop around it stays the same.
One possible output
| Line | What it shows |
|---|---|
| budget 2500, enough 3: accepted 2, rejected 1, spent 2750, stopped done | The third reply broke its limit, so spent passed the budget |
| rejected: b3 (over its limit) | The reason is kept |
| budget 1500, enough 3: accepted 1, rejected 0, spent 700, stopped budget | The second brief might not fit |
| budget 4000, enough 2: accepted 2, rejected 0, spent 1550, stopped enough | Two good results were enough |
| Summary: ... | A sentence that differs every run |
The cells students fill in are left blank in this preview.