Writing the risks down so none is forgotten
A risk register is a table of what could go wrong with a feature, how likely it is and what you will do about it.
A team that only talks about risks forgets some. A register is written down, so you can check it for gaps and rank the worst first.
By the end of this lesson
Note
I can write down the risks of an AI feature so that nothing is left out.
Key words
| Word | What it means |
|---|---|
| Risk | something that could go wrong for a person |
| Likelihood | how likely it is, from 1 to 3 |
| Impact | how bad it would be, from 1 to 3 |
| Mitigation | what you do to make a risk smaller |
| Area | a part of the data journey, such as storage |
The cells students fill in are left blank in this preview.
Each risk has a score and a mitigation
Give each risk a likelihood and an impact, each from 1 to 3. The score is the two multiplied. A score of 2 or less is low, 3 or 4 is medium, and 6 or 9 is high.
Every row also needs a mitigation. A risk with no plan is only a worry.
Worked example: likelihood times impact
| Risk | Likelihood | Impact | Score | Level |
|---|---|---|---|---|
| Prompt holds an email | 3 | 2 | 6 | High |
| Log keeps full prompts | 2 | 3 | 6 | High |
| Backup keeps deleted data | 2 | 2 | 4 | Medium |
| Staff read every chat | 1 | 3 | 3 | Medium |
The cells students fill in are left blank in this preview.
A register can have gaps and bad rows
A row can be wrong: a likelihood of 5, or an empty mitigation. A register can also be incomplete. Check that each area of the data journey has at least one row.
Typical areas are collection, sending, storage, access, deletion and training, which means using the data to teach a model. An area with no row is a gap.
A common mistake
Warning
Many people think the score is a measurement. It is a judgement written as a number, and two people can differ. Use it to order the talk and never to prove a feature is safe. A short register may only be an incomplete one.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the register checker, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const required = [\"collection\", \"sending\", \"storage\", \"access\", \"deletion\", \"training\"];' 'const register = [' ' { risk: \"Prompt holds an email\", area: \"sending\", l: 3, i: 2, fix: \"Redact first\" },' ' { risk: \"Log keeps full prompts\", area: \"storage\", l: 2, i: 3, fix: \"Log a length only\" },' ' { risk: \"Backup keeps deleted data\", area: \"deletion\", l: 2, i: 2, fix: \"\" },' ' { risk: \"Staff read every chat\", area: \"access\", l: 1, i: 3, fix: \"Limit readers\" },' ' { risk: \"Vendor keeps prompts\", area: \"collection\", l: 0, i: 3, fix: \"Contract check\" },' '];' > register.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the register checker, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function level(s) { return s <= 2 ? \"low\" : s <= 4 ? \"medium\" : \"high\"; }' 'function problems(r) {' ' const out = [];' ' if (r.l < 1 || r.l > 3) out.push(\"likelihood out of range\");' ' if (r.i < 1 || r.i > 3) out.push(\"impact out of range\");' ' if (r.fix.trim().length === 0) out.push(\"no mitigation\");' ' return out;' '}' >> register.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the register checker, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function covered(area) {' ' return register.some(function (r) { return r.area === area; });' '}' 'async function main() {' ' const ranked = register.slice().sort(function (a, b) { return b.l * b.i - a.l * a.i; });' ' ranked.forEach(function (r) {' ' const s = r.l * r.i;' ' console.log(s + \" \" + level(s) + \" | \" + r.risk + \" | \" + JSON.stringify(problems(r)));' ' });' ' console.log(\"missing areas: \" + JSON.stringify(required.filter(function (a) { return covered(a) === false; })));' ' const q = \"Give one short mitigation for this risk: a backup keeps data after a deletion request.\";' ' const sample = \"Set a short backup period and tell the person when it clears.\";' ' const reply = await ask([{ role: \"user\", content: q }], sample, 60);' ' console.log(\"model said: \" + reply.text);' '}' 'main();' >> register.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the register checker
docker exec -w /home/node/project agent node register.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
| Line | What it shows |
|---|---|
| 6 high | Prompt holds an email | [] | The worst risk, with no problem |
| 4 medium | Backup keeps deleted data | ["no mitigation"] | A row with a missing part |
| 0 low | Vendor keeps prompts | ["likelihood out of range"] | A row with a bad number |
| missing areas: ["training"] | An area with no row |
| model said: ... | A sentence. The words differ each run |
The cells students fill in are left blank in this preview.