A log that helps and does not leak
A log is a record an app writes about what it did. When a reply is slow or wrong, the log is where you look.
A careless log copies every prompt and every email into a file that many people can read. The aim is to keep enough to fix things and no more.
By the end of this lesson
Note
I can log enough to debug a request and no more than I should.
Key words
| Word | What it means | Example |
|---|---|---|
| Log record | one line a program writes about one event | |
| Request id | a label that ties every record of one request together | r-001 |
| Hash | a short code made from a value that cannot be turned back | |
| Salt | a secret word mixed in before hashing | |
| Truncate | to cut a value down to a few characters |
The cells students fill in are left blank in this preview.
Keep what you need to debug
Ask what you will use the log for. To fix a slow reply you need the request id, the time, the model and the length. You do not need the words.
A record without the words can still be joined to its request. The request id is the key that joins them.
Worked example: one record, before and after
| Field | Raw record | Safe record | Why |
|---|---|---|---|
| request_id | r-001 | r-001 | It joins the log to the request |
| user_email | [email protected] | e217f9d7 | A salted hash still counts one user |
| message | Refund my order please | 22 (its length) | The words stay out of the log |
| ms | 420 | 420 | Needed to find slow replies |
The cells students fill in are left blank in this preview.
A hash hides the value, not the person
A hash turns an email into a short code. The same email always gives the same code, so you can still count users. Nobody can turn the code back by reading it. The Run truncates it to 8 characters, which is enough for a small log.
A salt makes the code harder to guess. Also decide who may read the log. A log nobody needs to read should not exist.
A common mistake
Warning
Many people think a hash is anonymous. It is not. Anyone with a list of emails can hash them all and match the codes. A hash is a pseudonym, and the log is still personal data. Use a salt, and keep the salt away from the log.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the safe logger, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const crypto = require(\"crypto\");' 'const raw = [' ' { request_id: \"r-001\", user_email: \"[email protected]\", message: \"Refund my order please\", model: \"lab\", ms: 420 },' ' { request_id: \"r-002\", user_email: \"[email protected]\", message: \"Ring me on 07700900123\", model: \"lab\", ms: 380 },' ' { request_id: \"r-003\", user_email: \"[email protected]\", message: \"Where is my parcel\", model: \"lab\", ms: 910 },' '];' > logsafe.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the safe logger, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function hash8(v) {' ' return crypto.createHash(\"sha256\").update(\"lab-salt:\" + v).digest(\"hex\").slice(0, 8);' '}' 'function safe(r) {' ' return { request_id: r.request_id, user: hash8(r.user_email), length: r.message.length, model: r.model, ms: r.ms };' '}' 'function holdsEmail(r) {' ' return Object.values(r).some(function (v) { return typeof v === \"string\" && v.includes(\"@\"); });' '}' >> logsafe.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the safe logger, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' const out = raw.map(safe);' ' raw.forEach(function (r, i) {' ' console.log(r.request_id + \" raw holds email: \" + holdsEmail(r) + \" | safe holds email: \" + holdsEmail(out[i]));' ' console.log(\" \" + JSON.stringify(out[i]));' ' });' ' console.log(\"r-001 and r-003 same user: \" + (out[0].user === out[2].user));' ' console.log(\"r-001 and r-002 same user: \" + (out[0].user === out[1].user));' ' const q = \"In one short sentence, why log a message length and not the message?\";' ' const sample = \"A length helps you debug slow replies and holds nothing about the person.\";' ' const reply = await ask([{ role: \"user\", content: q }], sample, 60);' ' console.log(\"model said: \" + reply.text);' '}' 'main();' >> logsafe.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the safe logger
docker exec -w /home/node/project agent node logsafe.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
| Line | What it shows |
|---|---|
| r-001 raw holds email: true | safe holds email: false | The safe record has no email |
| {"request_id":"r-001","user":"e217f9d7",...} | The hash and the length, nothing else |
| r-001 and r-003 same user: true | The same person still joins |
| r-001 and r-002 same user: false | Two people stay apart |
| model said: ... | A sentence. The words differ each run |
The cells students fill in are left blank in this preview.