A diff can hide a deleted test
Reviewing a diff means reading the changes an AI agent made to your project and deciding whether to keep them. It matters because an agent can change several files at once, and one deleted test is easy to miss among a hundred lines you skim.
The review is your job, whoever wrote the change.
By the end of this lesson
Note
I can read an agent's change set and decide to keep, change or discard it.
Key words
| Word | What it means |
|---|---|
| Diff | the lines a change removes and adds |
| Hunk | one block of changed lines in a diff |
| Change set | every file a change touches |
| Scope | what the request asked to change |
| Discard | to throw a change away and ask again |
The cells students fill in are left blank in this preview.
List the files before you read the lines
Start with which files the change set touches. Compare that list with what you asked for.
A request to add one function should touch the file that holds it, and perhaps its tests. A change to a settings file or a script is outside the scope of the request. Question it before you read a single line.
One possible request
Tip
Add a function staffPrice(name) to tuck.js that gives the price with 10 percent off, rounded down. Change nothing else. Do not edit the tests or package.json unless you tell me why first.
Write the original code
docker exec -w /home/node/project agent sh -c "mkdir -p diff-demo/before && printf '%s\n' 'const PRICES = { crisps: 60, apple: 35 };' 'function price(name) {' ' return PRICES[name];' '}' 'module.exports = { price };' > diff-demo/before/tuck.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the original tests
docker exec -w /home/node/project agent sh -c "mkdir -p diff-demo/before && printf '%s\n' 'const test = require(\"node:test\");' 'const assert = require(\"node:assert\");' 'const { price } = require(\"./tuck\");' 'test(\"crisps cost 60\", () => assert.strictEqual(price(\"crisps\"), 60));' 'test(\"apple costs 35\", () => assert.strictEqual(price(\"apple\"), 35));' > diff-demo/before/tuck.test.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the original test command
docker exec -w /home/node/project agent sh -c "mkdir -p diff-demo/before && printf '%s\n' '{ \"scripts\": { \"test\": \"node --test\" } }' > diff-demo/before/package.json"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
An agent's change to the code
docker exec -w /home/node/project agent sh -c "cp -r diff-demo/before diff-demo/after && sed -i \"s/crisps: 60/crisps: 55/\" diff-demo/after/tuck.js && sed -i \"s/^module.exports = { price };/function staffPrice(name) { return Math.floor(PRICES[name] * 0.9); }\nmodule.exports = { price, staffPrice };/\" diff-demo/after/tuck.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
An agent's change to the tests
docker exec -w /home/node/project agent sh -c "sed -i \"/crisps cost/d\" diff-demo/after/tuck.test.js && sed -i \"s/{ price }/{ price, staffPrice }/\" diff-demo/after/tuck.test.js && printf '%s\n' 'test(\"staff price runs\", () => staffPrice(\"apple\"));' >> diff-demo/after/tuck.test.js && sed -i \"s/node --test/echo ok/\" diff-demo/after/package.json"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
List the files that differ
docker exec -w /home/node/project agent diff -rq diff-demo/before diff-demo/afterShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Read each hunk against the request
A diff marks removed lines with a minus and added lines with a plus. Each block is a hunk.
For every hunk ask whether the request needed it. Pure additions are the easy ones. Removed lines are the ones to read twice: a removed line is something that worked until now.
Show every hunk
docker exec -w /home/node/project agent sh -c "diff -U1 --label before/tuck.js --label after/tuck.js diff-demo/before/tuck.js diff-demo/after/tuck.js; diff -U1 --label before/tuck.test.js --label after/tuck.test.js diff-demo/before/tuck.test.js diff-demo/after/tuck.test.js; diff -U1 --label before/package.json --label after/package.json diff-demo/before/package.json diff-demo/after/package.json; true"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Reading the hunks
| File | What changed | Asked for? |
|---|---|---|
| tuck.js | The crisps price goes from 60 to 55 | No |
| tuck.js | A staffPrice function is added and exported | Yes |
| tuck.test.js | The import gains staffPrice | Only for the new test |
| tuck.test.js | The crisps test is removed | No |
| tuck.test.js | A test is added that checks nothing | Weak |
| package.json | The test command becomes echo ok | No |
The cells students fill in are left blank in this preview.
Keep, change or discard
Keep a change that does what you asked and nothing more. Change it when most is right: ask for the same request again with the extra edits ruled out. Discard it when it is bigger than the request or you cannot explain a hunk.
Run the tests yourself, with the test command as it was before the change. Never trust a report that the agent produced.
A common mistake
Warning
Many people think that if the tests pass, the change is safe. Here the tests could pass because the test that noticed the price was deleted and the test command was replaced. Passing tests only count when they are the same tests.
Try it on a change of your own
The lab's own job is the shop's fault, and this lesson leaves it alone. It is a good time to try the ideas on a different change.
Ask an agent for a small addition of your own, such as a new function. Before you save anything, list the files it touched, read each removed line and decide: keep, change or discard.