Few-shot prompting shows the model what you mean
Few-shot prompting means putting a few worked examples in the prompt. The model copies the pattern it sees, so the examples teach it your labels and your style.
It also means the examples can mislead. A prompt with four good comments and one bad one tells the model that most comments are good.
By the end of this lesson
Note
I can pick examples that show a model what I mean.
Key words
| Word | What it means |
|---|---|
| Example | an input paired with the answer you want |
| Label | the answer an example is given |
| Class | one of the labels a model may choose |
| Balance | each class shown about as often |
| Few-shot | a prompt with a few examples |
| Zero-shot | a prompt with no examples |
The cells students fill in are left blank in this preview.
How many examples does a prompt need?
With no examples a model uses its general sense of the task. One example fixes the format. Two or three show the range. Past five or six, each extra example adds tokens and usually little else.
Choose examples that differ from each other. Three near copies teach no more than one.
The same labels, balanced and not
| Set | good | bad | What the model learns |
|---|---|---|---|
| Unbalanced | 4 | 1 | Most comments are good, so say good |
| Balanced | 2 | 2 | Both labels are common |
| Missing a class | 3 | 0 | A bad comment does not exist |
The cells students fill in are left blank in this preview.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the example counter, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const examples = [' ' { text: \"The bus was on time\", label: \"good\" },' ' { text: \"My bag was left behind\", label: \"bad\" },' ' { text: \"Lovely driver today\", label: \"good\" },' ' { text: \"Quick and clean trip\", label: \"good\" },' ' { text: \"The seat was a bit wet\", label: \"good\" },' '];' > ex2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the example counter, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function counts(list) {' ' const out = {};' ' for (const e of list) out[e.label] = (out[e.label] || 0) + 1;' ' return out;' '}' 'function buildPrompt(list, text) {' ' const blocks = list.map(function (e) { return \"Text: \" + e.text + \"\nLabel: \" + e.label; });' ' blocks.push(\"Text: \" + text + \"\nLabel:\");' ' return \"Label each bus comment good or bad.\n\n\" + blocks.join(\"\n\n\");' '}' >> ex2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the example counter, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' const c = counts(examples);' ' console.log(\"counts: good \" + c.good + \", bad \" + c.bad);' ' for (const name of Object.keys(c)) {' ' if (c[name] * 2 > examples.length) console.log(\"warning: \" + name + \" is more than half of the examples\");' ' }' ' const prompt = buildPrompt(examples, \"The driver was rude\");' ' const reply = await ask([{ role: \"user\", content: prompt }], \"bad\", 5);' ' console.log(\"model: \" + reply.text);' '}' 'main();' >> ex2.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run and count the labels
docker exec -w /home/node/project agent node ex2.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What the Run prints
The first line counts the labels: good 4, bad 1. The second warns that good is more than half of the examples. That is the check that would catch a set like this before it reaches a model.
The last line is one short reply. With no model it prints a Recorded sample first.
A common mistake
Warning
Many people think the order of examples does not matter. Models often lean on the last example most. If it is a bad one, the new input is more likely to be called bad. Mix the classes so no class always comes last.