Better on average and still unsafe
An agent eval suite joins the end state, the path and repeated runs into one gate that a change must pass. It matters because each check alone lets a different bad change through.
The new shop agent scores 77 percent against 61. Yet one case it must never fail now fails sometimes.
By the end of this lesson
Note
I can put trajectory, state and repeated runs into one gate.
Key words
| Word | What it means |
|---|---|
| Gate | a pass mark fixed before the run |
| Must-pass case | a case that may never fail a run |
| Step growth | how many more steps the new version may take |
| Exit code | the number a script returns, 0 for success |
| Suite | the checks run together on every change |
The cells students fill in are left blank in this preview.
Each gate catches a different change
Set every gate before you look at the result. One gate says the end state is right often enough. One says the steps have not grown too far. One names the must-pass cases. One says the change is not clearly worse.
The steps gate limits step growth. It is a cost gate for an agent, since every step is paid for.
Worked example: four gates for the shop agent
| Gate | Rule | Catches |
|---|---|---|
| State rate | at least 70 percent of runs end right | a version that fails too often |
| Steps | no more than 20 percent more steps | a version that is right and slow |
| Must pass | case 4 passes in every run | a break in what matters most |
| Not worse | the sign test does not say worse | a change that fixed few and broke many |
The cells students fill in are left blank in this preview.
A gate ends in an exit code
Each gate adds its name to a list of failures. The suite prints the list and returns an exit code: 0 when the list is empty and 1 when it is not.
The lab has no CI, short for continuous integration: a service that runs checks on every change. There, a real suite would run this script and stop the release on a 1.
Write the recorded runs
docker exec -w /home/node/project agent sh -c "printf '%s\n' '// Three recorded runs of each case: [state is right (1 or 0), steps taken].' 'const oldVersion = [' ' [[1, 4], [1, 4], [1, 5]], [[1, 5], [0, 6], [1, 5]], [[0, 6], [0, 7], [1, 6]],' ' [[1, 4], [1, 4], [1, 4]], [[1, 5], [1, 5], [0, 5]], [[0, 7], [0, 7], [0, 6]],' '];' 'const newVersion = [' ' [[1, 4], [1, 4], [1, 4]], [[1, 5], [1, 5], [1, 6]], [[1, 6], [0, 7], [1, 6]],' ' [[1, 5], [0, 5], [1, 5]], [[1, 5], [1, 5], [1, 5]], [[0, 7], [1, 7], [0, 7]],' '];' 'const settings = { minState: 70, stepGrowth: 20, must: [3] };' 'module.exports = { oldVersion, newVersion, settings };' > runs.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the suite, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { oldVersion, newVersion, settings } = require(\"./runs\");' 'function states(version) {' ' return version.map(function (runs) {' ' return runs.filter(function (run) { return run[0] === 1; }).length;' ' });' '}' 'function stateRate(version) {' ' const total = states(version).reduce(function (a, b) { return a + b; }, 0);' ' return Math.floor(total * 100 / (version.length * version[0].length));' '}' 'function steps(version) {' ' let sum = 0;' ' version.forEach(function (runs) { runs.forEach(function (run) { sum = sum + run[1]; }); });' ' return sum;' '}' > suite.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the suite, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function signP(better, worse) {' ' const n = better + worse;' ' if (n === 0) return 1;' ' let total = 0;' ' let ways = 1;' ' for (let i = 0; i <= Math.min(better, worse); i = i + 1) {' ' total = total + ways;' ' ways = ways * (n - i) / (i + 1);' ' }' ' return Math.min(1, 2 * total / Math.pow(2, n));' '}' 'const before = states(oldVersion);' 'const after = states(newVersion);' 'let better = 0;' 'let worse = 0;' 'before.forEach(function (n, at) {' ' if (after[at] > n) better = better + 1;' ' if (after[at] < n) worse = worse + 1;' '});' >> suite.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the suite, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const failures = [];' 'const rate = stateRate(newVersion);' 'console.log(\"State rate \" + rate + \" percent, needs \" + settings.minState);' 'if (rate < settings.minState) failures.push(\"state rate\");' 'console.log(\"Steps \" + steps(newVersion) + \" against \" + steps(oldVersion) + \", limit \" + settings.stepGrowth + \" percent more\");' 'if (steps(newVersion) * 100 > steps(oldVersion) * (100 + settings.stepGrowth)) failures.push(\"steps\");' 'const broken = settings.must.filter(function (at) { return after[at] < newVersion[at].length; });' 'console.log(\"Must-pass cases that failed a run: \" + broken.map(function (at) { return at + 1; }).join(\", \"));' 'if (broken.length > 0) failures.push(\"must pass\");' >> suite.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the suite, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const p = signP(better, worse);' 'console.log(\"Better \" + better + \", worse \" + worse + \", p \" + p.toFixed(3));' 'if (p < 0.05 && worse > better) failures.push(\"worse\");' 'console.log(failures.length === 0 ? \"All gates passed\" : \"Gates failed: \" + failures.join(\", \"));' 'process.exit(failures.length === 0 ? 0 : 1);' >> suite.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the suite and read its exit code
docker exec -w /home/node/project agent node suite.js; echo exit code $?Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the suite prints
| Line | What it shows |
|---|---|
| State rate | 77 percent, needs 70: passes |
| Steps | 98 against 95, limit 20 percent: passes |
| Must-pass cases | case 4 failed a run |
| Better and worse | better 4, worse 1, p 0.375: passes |
| Result | Gates failed: must pass |
| Exit code | 1 |
The cells students fill in are left blank in this preview.
Try it
Tip
In runs.js change the second run of the new fourth case from [0, 5] to [1, 5], then run again. The must-pass gate passes and the exit code is 0.
A common mistake
Warning
Many people loosen a gate when a good change fails it. A gate moved after a failure no longer protects anything. If the gate was wrong, change it in its own review with its reason written down, and never in the same change that failed it.