How often does an answer invent facts?
A hallucination is a claim in an answer that its source does not support. A help bot that says a library opens at 8 when it opens at 9 has hallucinated.
To manage this, you measure it as a rate, and you say how sure you are of that rate.
By the end of this lesson
Note
I can measure how often an answer says what its source does not.
Key words
| Word | What it means |
|---|---|
| Claim | one fact stated in an answer |
| Source | the text the answer should rely on |
| Unsupported | a claim the source does not back |
| Rate | unsupported claims as a share of all claims |
| Interval | the range a rate may really lie in |
The cells students fill in are left blank in this preview.
Check each claim against the source
Split an answer into claims, one fact each. For every claim, ask whether the source says it.
A simple first pass uses key words. A claim is supported when every key word it needs is in the source. It is unsupported when one is missing.
Worked example: four claims on one source
| Claim | Key words | In the source? | Verdict |
|---|---|---|---|
| It opens at 9 | opens, 9 | Both | Supported |
| It has 20 computers | computers, 20 | 20 is missing | Unsupported |
| Fines are 20p a day | 20p, day | Both | Supported |
| It opens on Sundays | sundays | Missing | Unsupported |
The cells students fill in are left blank in this preview.
The rate is a count over a count
Count the unsupported claims. Divide by all the claims and write it as a percent. Three unsupported claims out of eight is 37.5 percent.
Count claims, not answers. One answer with five claims, one of them false, is one bad claim in five.
Resample to see how rough the rate is
Write the 40 claims as 0 for supported and 1 for unsupported. Draw 40 from that list at random, putting each back after it is drawn. Count the ones.
Repeat 200 times and sort the counts. The 6th and the 195th mark the range, which holds the middle 190 of the 200 counts. A fixed seed makes the draws repeat.
Worked example: 6 unsupported in 40
| Step | Result |
|---|---|
| Claims and unsupported | 40 and 6 |
| Rate | 15 percent |
| Lowest sorted count in the range | 2, which is 5 percent |
| Highest sorted count in the range | 10, which is 25 percent |
| What to report | 15 percent, range 5 to 25 |
The cells students fill in are left blank in this preview.
A common mistake
Warning
Many people think 6 unsupported out of 40 means the true rate is exactly 15 percent. It is only what this sample shows. A different 40 claims could give 10 or 20. Report the rate with its interval.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the rate and interval, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const source = \"The Harbour Library opens at 9 and closes at 5 on weekdays. It has 12 computers. Members may borrow 6 books for 3 weeks. A late fee is 20p a day.\";' 'const have = source.toLowerCase().split(\".\").join(\" \").split(\" \");' 'const claims = [' ' \"opens 9\", \"closes 5\", \"computers 12\", \"computers 20\",' ' \"borrow 6 books\", \"weeks 4\", \"20p day\", \"sundays\",' '].map(function (c) { return c.split(\" \"); });' > rate.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the rate and interval, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function supported(needs) {' ' return needs.every(function (w) { return have.indexOf(w) >= 0; });' '}' 'const bad = claims.filter(function (c) { return supported(c) === false; });' 'const flags = \"0001000001000001000000100000100000010000\".split(\"\").map(Number);' >> rate.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the rate and interval, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function interval(list, seed) {' ' const n = list.length;' ' const counts = [];' ' let s = seed;' ' for (let r = 0; r < 200; r++) {' ' let total = 0;' ' for (let k = 0; k < n; k++) { s = (s * 16807) % 2147483647; total += list[s % n]; }' ' counts.push(total);' ' }' ' counts.sort(function (a, b) { return a - b; });' ' return [counts[5] * 100 / n, counts[194] * 100 / n];' '}' >> rate.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the rate and interval, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' console.log(\"unsupported: \" + bad.length + \" of \" + claims.length + \" (\" + (bad.length * 100 / claims.length) + \" percent)\");' ' const ones = flags.filter(function (f) { return f === 1; }).length;' ' const span = interval(flags, 12345);' ' console.log(\"bigger test: \" + ones + \" of \" + flags.length + \" = \" + (ones * 100 / flags.length) + \" percent\");' ' console.log(\"interval: \" + span[0] + \" to \" + span[1] + \" percent\");' ' const q = \"The source says the library opens at 9. Is this claim supported: it opens at 8? One short sentence.\";' ' const sample = \"No, the source says it opens at 9, so the claim is not supported.\";' ' const reply = await ask([{ role: \"user\", content: q }], sample, 60);' ' console.log(\"model said: \" + reply.text);' '}' 'main();' >> rate.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the rate and interval
docker exec -w /home/node/project agent node rate.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
| Line | What it shows |
|---|---|
| unsupported: 3 of 8 (37.5 percent) | The keyword check on eight claims |
| bigger test: 6 of 40 = 15 percent | A recorded list of 40 claims |
| interval: 5 to 25 percent | The resampled range |
| model said: ... | One real check from the lab. A recorded sample if no model answers |
The cells students fill in are left blank in this preview.