Removing personal data and what it breaks
Redacting means taking personal data out of a message before it goes anywhere else. A support app can redact a phone number before it asks a model for a reply.
The catch is that the reply may need that number. How you redact decides whether you can put it back.
By the end of this lesson
Note
I can remove personal data and say what that breaks.
Key words
| Word | What it means | Example |
|---|---|---|
| Redact | to take personal data out of text | |
| Mask | to replace a value with its kind | [EMAIL] |
| Token | a stand-in that can be swapped back | [EMAIL_1] |
| Map | the private list of which token is which value | |
| Restore | to put the real values back |
The cells students fill in are left blank in this preview.
There are three ways to redact
A mask replaces a value with its kind, such as [EMAIL]. Two different emails both become [EMAIL], so you cannot tell them apart or put them back.
A token is numbered, such as [EMAIL_1], and it is reversible. You keep a private map of tokens and values and never send it. Destroying a value means it is gone for good and no map is kept.
Worked example: one email, three ways
| Method | What the model sees | Can the reply name it? | Can you restore it? |
|---|---|---|---|
| Mask | [EMAIL] | No | No |
| Token | [EMAIL_1] | Yes, as a token | Yes, from your map |
| Destroy | Nothing | No | No, by design |
The cells students fill in are left blank in this preview.
Redacting can break the reply
If the model never sees the email, it cannot write it. A reply that says "we will write to your address" is fine. A reply that must repeat the address needs a token.
Tokens bring a new risk. The model may drop one or change it. Restore only the tokens that come back, and check that none is left.
A common mistake
Warning
Many people think tokens make data anonymous. They do not. You still hold the map, so you can turn them back into people. In UK and EU data protection words this is pseudonymised data, and it is still personal data. This is a plain summary, not legal advice.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the redactor, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const text = \"Please write to [email protected] or ring 07700900123. Thanks, Ana.\";' 'const counts = {};' 'const map = {};' 'function clean(word) {' ' let w = word;' ' while (w.length > 0 && \".,;:?()\".includes(w[w.length - 1])) w = w.slice(0, -1);' ' return w;' '}' > redact.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the redactor, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function kindOf(w) {' ' const at = w.indexOf(\"@\");' ' if (at > 0 && w.indexOf(\".\", at) > at + 1) return \"EMAIL\";' ' if (w.length === 11 && w.startsWith(\"07\") && Number.isInteger(Number(w))) return \"PHONE\";' ' return \"\";' '}' 'function mask(t) {' ' return t.split(\" \").map(function (word) {' ' const w = clean(word);' ' return kindOf(w).length === 0 ? word : \"[\" + kindOf(w) + \"]\" + word.slice(w.length);' ' }).join(\" \");' '}' >> redact.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the redactor, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function tokenise(t) {' ' const known = {};' ' return t.split(\" \").map(function (word) {' ' const w = clean(word);' ' const kind = kindOf(w);' ' if (kind.length === 0) return word;' ' if (known[w] === undefined) {' ' counts[kind] = (counts[kind] || 0) + 1;' ' known[w] = \"[\" + kind + \"_\" + counts[kind] + \"]\";' ' map[known[w]] = w;' ' }' ' return known[w] + word.slice(w.length);' ' }).join(\" \");' '}' >> redact.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the redactor, part 4
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function restore(t) {' ' let out = t;' ' Object.keys(map).forEach(function (k) { out = out.split(k).join(map[k]); });' ' return out;' '}' 'async function main() {' ' const safe = tokenise(text);' ' console.log(\"masked: \" + mask(text));' ' console.log(\"tokenised: \" + safe);' ' console.log(\"map kept here: \" + JSON.stringify(map));' ' const q = \"Write one short reply to this message. Copy each token in square brackets exactly. \" + safe;' ' const sample = \"Thank you. We will write to [EMAIL_1] and ring [PHONE_1] today.\";' ' const reply = await ask([{ role: \"user\", content: q }], sample, 80);' ' console.log(\"reply: \" + reply.text);' ' console.log(\"restored: \" + restore(reply.text));' '}' 'main();' >> redact.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run the redactor
docker exec -w /home/node/project agent node redact.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
| Line | What it shows |
|---|---|
| masked: ... [EMAIL] ... [PHONE] ... | The kinds only; nothing can be put back |
| tokenised: ... [EMAIL_1] ... [PHONE_1] ... | Numbered stand-ins |
| map kept here: {...} | The private list, never sent |
| reply: ... | A reply with tokens. Recorded when no model answers |
| restored: ... | The tokens swapped back to the invented values |
The cells students fill in are left blank in this preview.