How to make a tamper-evident log
An audit trail is a log of who changed what and when. A log that anyone can edit is not evidence, because nobody can tell if it was changed.
It matters because a decision about an AI change is only as good as its record. A hash chain makes any edit to the record show.
By the end of this lesson
Note
I can keep a log that shows if it was changed.
Key words
| Word | What it means | Example |
|---|---|---|
| Append only | new entries are added and old ones never edited | |
| Hash | a short fingerprint of some text | 1a47e90b |
| Chain | entries that each hold the one before's hash | |
| Tamper | to change a record without leave | |
| Anchor | the latest hash, kept somewhere else |
The cells students fill in are left blank in this preview.
A hash is a fingerprint of text
A hash turns any text into a short code. The same text always gives the same code. Change one letter and the code changes completely.
The hash in this lesson is small and made for teaching. It is not secure. A real log uses a strong hash from a library, and the idea is the same.
Worked example: three fingerprints
| Text | Hash |
|---|---|
| (nothing) | 811c9dc5 |
| a | e40c292c |
| abc | 1a47e90b |
The cells students fill in are left blank in this preview.
A chain links each entry to the last
Each entry holds its text, the hash of the entry before, and its own hash. Its own hash is made from the previous hash, a bar and its text.
Edit entry 2 and its own hash no longer matches its text. Work out a new hash to hide that, and entry 3 still holds the old one, so the next link breaks instead.
Worked example: five entries
| Entry | Text | Previous hash | Own hash |
|---|---|---|---|
| 0 | model pinned to chat-v3 | 00000000 | ae34fd31 |
| 1 | canary at 5 percent | ae34fd31 | 9f28baa1 |
| 2 | canary at 20 percent | 9f28baa1 | e224ba61 |
| 3 | promoted to all users | e224ba61 | 129fa366 |
| 4 | old version retired | 129fa366 | 149f5379 |
The cells students fill in are left blank in this preview.
A common mistake
Warning
Many people think a hash chain stops people changing the log. It does not. It only shows a change. Someone who can write the whole file can rebuild every hash. Keep the latest hash, the anchor, somewhere they cannot reach, and compare it.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the audit script, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const START = \"00000000\";' 'function hash(text) {' ' let x = 2166136261;' ' for (let i = 0; i < text.length; i++) {' ' x = x ^ text.charCodeAt(i);' ' x = Math.imul(x, 16777619) >>> 0;' ' }' ' return (\"00000000\" + x.toString(16)).slice(-8);' '}' > audit.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the audit script, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function add(chain, text) {' ' const prev = chain.length > 0 ? chain[chain.length - 1].hash : START;' ' return chain.concat([{ text: text, prev: prev, hash: hash(prev + \"|\" + text) }]);' '}' 'function firstBroken(chain) {' ' let prev = START;' ' for (let i = 0; i < chain.length; i++) {' ' const e = chain[i];' ' if (e.prev === prev && e.hash === hash(e.prev + \"|\" + e.text)) prev = e.hash;' ' else return i;' ' }' ' return -1;' '}' >> audit.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the audit script, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const notes = [\"model pinned to chat-v3\", \"canary at 5 percent\", \"canary at 20 percent\",' ' \"promoted to all users\", \"old version retired\"];' 'let chain = [];' 'notes.forEach(function (t) { chain = add(chain, t); });' 'const anchor = chain[4].hash;' 'const forged = chain.map(function (e) { return { text: e.text, prev: e.prev, hash: e.hash }; });' 'forged[2].text = \"canary at 90 percent\";' 'let again = [];' 'forged.forEach(function (e) { again = add(again, e.text); });' >> audit.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the audit script, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' console.log(\"hash of abc: \" + hash(\"abc\"));' ' console.log(\"first broken in the real log: \" + firstBroken(chain));' ' console.log(\"first broken after an edit: \" + firstBroken(forged));' ' console.log(\"rebuilt log is consistent: \" + (firstBroken(again) === -1));' ' console.log(\"rebuilt log matches the anchor: \" + (again[4].hash === anchor));' ' const q = \"In one short sentence, why is an edited log entry easy to spot in a hash chain?\";' ' const sample = \"Each entry holds a fingerprint of the one before, so one change breaks every link after it.\";' ' const reply = await ask([{ role: \"user\", content: q }], sample, 60);' ' console.log(\"model said: \" + reply.text);' '}' 'main();' >> audit.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the audit trail
docker exec -w /home/node/project agent node audit.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
| Line | What it shows |
|---|---|
| hash of abc: 1a47e90b | The same fingerprint every time |
| first broken in the real log: -1 | No broken link |
| first broken after an edit: 2 | The edit is found at entry 2 |
| rebuilt log is consistent: true | A full rebuild passes the link check |
| rebuilt log matches the anchor: false | The anchor still shows the change |
| model said: ... | A sentence from the lab. A recorded sample if no model answers |
The cells students fill in are left blank in this preview.