A person who clicks Approve without reading
Human oversight means a person checks what an AI feature is about to do, and can stop it. It matters because a check nobody reads is worse than none: it gives everyone a false sense of safety.
This lesson is about making the check real. Which actions need a person, what that person must see, and how to spot an approval that has become a habit.
By the end of this lesson
Note
I can design a check by a person that is real, not a habit.
Key words
| Word | What it means |
|---|---|
| Oversight | a person who may check and stop an action |
| Approval | a person saying yes to one action |
| Rubber stamp | an approval given without reading |
| Escalation | passing a case to a more senior person |
| Threshold | the size above which a person must decide |
The cells students fill in are left blank in this preview.
Choose what needs a person
A person cannot read every action, so choose. Ask a person when the action costs more than a threshold, or cannot be undone. Let the rest go ahead and log it.
A person who is asked about everything stops reading. Few, well-chosen questions get real attention.
Worked example: four actions, limit 100 euros
| Action | Money | Undoable | Who decides |
|---|---|---|---|
| Send a thank-you email | 0 | yes | Goes ahead |
| Refund order 114 | 40 | yes | Goes ahead |
| Refund order 207 | 450 | yes | A person: over the limit |
| Delete a customer file | 0 | no | A person: cannot be undone |
The cells students fill in are left blank in this preview.
A real check shows what the person needs
Show the person what is about to happen, how much it is, and what is not known. Do not show only the model's answer. Give a way to say no and a way to pass the case up.
Then measure. An approval that took one second was not read. If most approvals are quick and almost all say yes, the check has become a habit.
A common mistake
Warning
Many people think that having a person in the loop is enough. A person who approves 200 items an hour is not checking them. Count how long each approval takes and how many are refused, and treat a very fast, very high yes rate as a fault.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the oversight check, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' '' 'const LIMIT = 100; // euros: above this a person decides' 'const MIN_SECS = 5; // an approval faster than this was not read' 'const actions = [' ' { id: \"a1\", what: \"Send a thank-you email\", money: 0, undo: true },' ' { id: \"a2\", what: \"Refund order 114\", money: 40, undo: true },' ' { id: \"a3\", what: \"Refund order 207\", money: 450, undo: true },' ' { id: \"a4\", what: \"Delete a customer file\", money: 0, undo: false },' '];' 'function needsPerson(a) {' ' return a.money > LIMIT || a.undo === false;' '}' > oversight.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the oversight check, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' '// Recorded approvals: how many seconds each person took.' 'const seen = [' ' { id: \"r1\", secs: 31 },' ' { id: \"r2\", secs: 1 },' ' { id: \"r3\", secs: 45 },' ' { id: \"r4\", secs: 2 },' '];' >> oversight.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the oversight check, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' actions.forEach(function (a) {' ' console.log(a.id + \": \" + (needsPerson(a) ? \"ask a person\" : \"go ahead\"));' ' });' ' seen.forEach(function (r) {' ' console.log(r.id + \": \" + r.secs + \" s \" + (r.secs < MIN_SECS ? \"NOT READ\" : \"read\"));' ' });' ' const q = \"Write two short lines for a manager who must approve: Refund order 207, 450 euros. Give what, how much, and what is not known.\";' ' const r = await ask([{ role: \"user\", content: q }], \"Refund of 450 euros on order 207. The reason given is not checked and the customer history is not shown.\", 80);' ' console.log(\"One possible summary: \" + r.text);' '}' 'main();' >> oversight.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the oversight check
docker exec -w /home/node/project agent node oversight.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
One possible output
| Line | What it shows |
|---|---|
| a1: go ahead and a2: go ahead | Small and undoable |
| a3: ask a person and a4: ask a person | Over the limit, and cannot be undone |
| r2: 1 s NOT READ and r4: 2 s NOT READ | Approvals too fast to be real |
| One possible summary: ... | Your own call. It differs each run, or says Recorded sample |
The cells students fill in are left blank in this preview.