The description sounds right, but is it?
A model describes a photo: two cats, one red ball and a sign that says OPEN. It sounds sure. But a model can add a dog that is not there, or say three cats when there are two.
This lesson shows how to check each claim against what you know about the input.
By the end of this lesson
Note
I can check each claim in a model's description against known facts and decide whether to accept the reply.
Key words
| Word | What it means |
|---|---|
| Claim | one thing a reply says about the input |
| Evidence | the known fact that a claim can be checked against |
| Invented | a claim about something that is not in the input |
| Unchecked | a claim with no fact to check it against |
| Verdict | the result of checking one claim |
The cells students fill in are left blank in this preview.
Split a reply into claims
A reply is easier to check when you cut it into claims, each one small enough to be true or false. "Two cats and a red ball" is three claims.
You can only check a claim you have a fact for. The facts may come from the input itself, such as counting items in a grid, or from a person who knows the picture.
Four claims and their facts
| Claim | Fact held | Verdict |
|---|---|---|
| There are 2 cats | cats: 2 | supported |
| There are 3 cats | cats: 2 | wrong number |
| There is a dog | no dog listed | invented |
| The door is red | no colour for the door | unchecked |
The cells students fill in are left blank in this preview.
Different faults need different verdicts
A wrong number, a wrong colour and an invented thing are different faults. An invented thing is the worst, because the reply is about something that is not there.
So the decision follows the worst verdict. Any invented claim means reject. A wrong or unchecked claim means a person reviews it. Only a reply with nothing wrong is accepted.
A common mistake
Warning
Many people accept a reply because most of it is right. Four true claims do not make a fifth one true. Check every claim. One invented thing can be enough to reject the whole reply.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checking script, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const grid = [\"..R...\", \".B..R.\", \"R.....\", \"...G..\"];' 'const claims = [' ' { kind: \"count\", thing: \"R\", value: 3 },' ' { kind: \"count\", thing: \"B\", value: 2 },' ' { kind: \"count\", thing: \"Y\", value: 1 },' ' { kind: \"exists\", thing: \"G\" },' '];' > check5.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checking script, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function facts(g) {' ' const counts = {};' ' for (const row of g) {' ' for (const ch of row) {' ' if (ch === \".\") continue;' ' counts[ch] = (counts[ch] || 0) + 1;' ' }' ' }' ' return counts;' '}' >> check5.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checking script, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function verdict(counts, c) {' ' if (c.kind === \"exists\") return c.thing in counts ? \"supported\" : \"invented\";' ' if ((c.thing in counts) === false) return \"invented\";' ' return counts[c.thing] === c.value ? \"supported\" : \"wrong number\";' '}' >> check5.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checking script, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' const counts = facts(grid);' ' console.log(\"facts \" + JSON.stringify(counts));' ' for (const c of claims) {' ' const value = c.value === undefined ? \"\" : \" \" + c.value;' ' console.log(c.kind + \" \" + c.thing + value + \": \" + verdict(counts, c));' ' }' ' const q = \"In one short sentence, why should a description of a picture be checked against the picture?\";' ' const reply = await ask([{ role: \"user\", content: q }], \"A description can add things or get a number wrong, so check it against the picture.\", 40);' ' console.log(\"reply: \" + reply.text);' '}' 'main();' >> check5.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check four recorded claims
docker exec -w /home/node/project agent node check5.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The lines it prints
| Line | What it shows |
|---|---|
| facts {"R":3,"B":1,"G":1} | What the script counted in its grid of letters |
| count R 3: supported | The claim matches the count |
| count B 2: wrong number | The count was 1 |
| count Y 1: invented | There is no Y in the grid |
The cells students fill in are left blank in this preview.