Writing a task for a coding agent
A task for a coding agent is the written instruction the agent works from. The task decides what you get back, so a clear one saves hours of checking.
A coding agent is a program that uses a language model to read a repository, change files and run commands. "Make the shop better" can come back as forty changed files. A clear task comes back as a small change you can check.
By the end of this lesson
Note
I can write a task specification that a coding agent can finish and I can check.
Key words
| Word | What it means | Example |
|---|---|---|
| Coding agent | a program that reads code, edits files and runs commands with a model | |
| Task specification | written instructions for one piece of work | |
| Scope | the part of the work that is included | |
| Done when | a check that shows the work is finished | node --test passes |
| Out of scope | what the work must not touch |
The cells students fill in are left blank in this preview.
A task for an agent has five parts
The goal says what changes, in one sentence. The scope says which part of the work is included. The files list says where the agent may write.
Done when names a check you can run, such as a test command. Out of scope says what must stay as it is. An agent rarely stops to ask, so each part answers a question it would guess at.
Worked example: vague task, clear task
| Part | Vague task | Clear task |
|---|---|---|
| Goal | Make the shop better | Add a 10 percent student discount to total |
| Scope | Not said | total only |
| Files | Not said | shop.js and shop.test.js |
| Done when | Not said | node --test passes with 2 new tests |
| Out of scope | Not said | the prices and asPounds |
The cells students fill in are left blank in this preview.
A common mistake
Warning
Many people think an agent will ask when a task is unclear. Most agents guess and carry on, so a gap becomes a choice you did not make. Write the out of scope part and the done when check before you start the agent.
A script checks a task first
A checklist catches the usual gaps in a second. This script holds three recorded specs, A, B and C. It looks for missing parts, vague words such as "better", more than three files and a done when with no test.
Nothing here starts the agent. The Run checks the words only, so its first three lines are the same for everyone.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the spec checker, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const VAGUE = [\"better\", \"improve\", \"clean\", \"tidy\", \"nicer\", \"properly\"];' 'const PARTS = [\"goal\", \"scope\", \"files\", \"done_when\", \"out_of_scope\"];' 'const specs = {' ' \"A\": { goal: \"Make the shop code better and tidy it up\" },' ' \"B\": { goal: \"Add a 10 percent student discount to total\", scope: \"total only\",' ' files: [\"shop.js\", \"shop.test.js\", \"prices.js\", \"README.md\"], done_when: \"node --test passes\",' ' out_of_scope: \"the prices\" },' ' \"C\": { goal: \"Add a 10 percent student discount to total\", scope: \"total only\",' ' files: [\"shop.js\", \"shop.test.js\"], done_when: \"node --test passes with 2 new tests\",' ' out_of_scope: \"the prices and asPounds\" },' '};' > spec.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the spec checker, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function problems(s) {' ' const out = PARTS.filter(function (p) { return s[p] === undefined || s[p].length === 0; })' ' .map(function (p) { return p + \" missing\"; });' ' if (s.goal) {' ' const words = s.goal.toLowerCase().split(\" \");' ' VAGUE.forEach(function (v) { if (words.indexOf(v) >= 0) out.push(\"vague: \" + v); });' ' }' ' if (s.files && s.files.length > 3) out.push(\"too many files\");' ' if (s.done_when && s.done_when.indexOf(\"test\") < 0) out.push(\"done_when has no test\");' ' return out;' '}' >> spec.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the spec checker, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' Object.keys(specs).forEach(function (name) {' ' const found = problems(specs[name]);' ' console.log(\"spec \" + name + \": \" + (found.length === 0 ? \"ready\" : \"needs work \" + JSON.stringify(found)));' ' });' ' const q = \"In one short sentence, why does a task for a coding agent need a done-when check?\";' ' const sample = \"Without a check, the agent cannot tell when to stop and you cannot tell if it worked.\";' ' const reply = await ask([{ role: \"user\", content: q }], sample, 60);' ' console.log(\"model said: \" + reply.text);' '}' 'main();' >> spec.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the spec checker
docker exec -w /home/node/project agent node spec.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
| Line | What it shows |
|---|---|
| spec A: needs work [...] | Four parts missing, and two vague words |
| spec B: needs work ["too many files"] | Four files is too wide for one task |
| spec C: ready | Every part present and the check is a test |
| model said: ... | A sentence from the lab. A Recorded sample line if no model answers |
The cells students fill in are left blank in this preview.