A user uploads a file called holiday.png
A user adds a file called holiday.png to your chat box. You cannot know what is inside it. It could be a photo, or a program with a picture name.
Sending an image to a model safely means checking it before it leaves your server. This lesson shows the checks.
By the end of this lesson
Note
I can build a request that carries an image, and refuse a file that breaks a rule before it is sent.
Key words
| Word | What it means | Example |
|---|---|---|
| Base64 | a way to write bytes as plain text letters and digits | |
| Media type | a label that says what kind of file it is | image/png |
| First bytes | the start of a file, which says what it really is | |
| Payload | the data a request carries | |
| Limit | the most the service accepts |
The cells students fill in are left blank in this preview.
A picture travels as text
A request is usually text, but a picture is bytes. Base64 turns every 3 bytes into 4 plain characters, so the request stays text.
This has a price. The text is one third longer than the bytes. So a limit on the bytes is not a limit on the request.
How base64 grows a file
| Bytes | Characters | Why |
|---|---|---|
| 3 | 4 | One group of 3 becomes 4 |
| 4 | 8 | A second group is started |
| 28 | 40 | Ten groups of 4 |
| 3,000 | 4,000 | About one third more |
The cells students fill in are left blank in this preview.
Trust the first bytes, not the name
A name is only what somebody typed. The first bytes of a file are fixed by its format. A PNG always starts with the same 8 bytes, and a JPEG with 3.
So decide the media type by reading the first bytes. If the name disagrees, refuse the file or at least say so.
A common mistake
Warning
Many people trust the file name and the type the browser reports. Both are typed by the sender. Check the first bytes on your own server, because a program named holiday.png would pass a check on the name.
Check first, then send
Run every check before a single byte leaves. Collect all the problems in a list, in a fixed order, so the user can fix everything at once. Only a file with no problems is encoded and put in the payload.
A refused call still costs time, and the model never sees a file you did not check.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checking script, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const head = [137, 80, 78, 71, 13, 10, 26, 10];' 'const png = Buffer.concat([Buffer.from(head), Buffer.alloc(40, 7)]);' 'const fake = Buffer.concat([Buffer.from(\"MZ\"), Buffer.alloc(40, 7)]);' 'const huge = Buffer.concat([Buffer.from(head), Buffer.alloc(142, 7)]);' 'const LIMIT = 100;' > send2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checking script, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function sniff(b) {' ' if (b.length >= 8 && b[0] === 137 && b[1] === 80 && b[2] === 78 && b[3] === 71) return \"png\";' ' if (b.length >= 3 && b[0] === 255 && b[1] === 216 && b[2] === 255) return \"jpeg\";' ' return \"unknown\";' '}' 'function check(b) {' ' const problems = [];' ' if (b.length > LIMIT) problems.push(\"too big\");' ' if (sniff(b) === \"unknown\") problems.push(\"type not allowed\");' ' return problems;' '}' >> send2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the checking script, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' const list = [[\"cat.png\", png], [\"notes.png\", fake], [\"huge.png\", huge]];' ' for (const item of list) {' ' const b = item[1];' ' console.log(item[0] + \": \" + sniff(b) + \", \" + b.length + \" bytes, base64 \" + b.toString(\"base64\").length + \" chars\");' ' console.log(\" problems: \" + JSON.stringify(check(b)));' ' }' ' const part = { type: \"image\", media_type: \"image/png\", data: png.toString(\"base64\").slice(0, 12) + \"...\" };' ' console.log(JSON.stringify({ messages: [{ role: \"user\", content: [{ type: \"text\", text: \"Describe it.\" }, part] }] }));' ' const q = \"Write one short, kind sentence telling a user their picture was too big to send.\";' ' const reply = await ask([{ role: \"user\", content: q }], \"That picture is too big to send, so please choose a smaller one.\", 40);' ' console.log(\"reply: \" + reply.text);' '}' 'main();' >> send2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Check three files
docker exec -w /home/node/project agent node send2.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
The lines it prints
| Line | What it shows |
|---|---|
| cat.png: png, 48 bytes, base64 64 chars | Real PNG bytes, and 48 bytes became 64 characters |
| notes.png: unknown ... type not allowed | The name says png, the bytes do not |
| huge.png: png, 150 bytes ... too big | Right type, over the limit of 100 |
| {"messages":[...]} | The request body, with the data cut short |
The cells students fill in are left blank in this preview.