The right page is fourth
Reranking means taking the short list a search returned and putting it in a better order with a closer look. It matters because the first pass is fast and rough, and the page that answers is often not on top.
A user asks how to get money back. The refund policy page is fourth, below a page about paying by card. A second look moves it to the top.
By the end of this lesson
Note
I can reorder a short list of results with a second pass and say whether it was worth its cost.
Key words
| Word | What it means |
|---|---|
| First pass | the fast search over the whole index |
| Second pass | a slower look at the first pass's top few |
| Reranker | what scores the short list again |
| Candidate | a result the second pass may reorder |
| Recall ceiling | the best the second pass can do, set by the first |
| Hit rate at k | the share of questions whose source is in the top k |
The cells students fill in are left blank in this preview.
A wide cheap pass, then a narrow one
Looking closely at every page is too slow. So search works in two passes. The first pass is cheap and wide. It cuts millions of pages to a few dozen. The second pass is slow and narrow. It reads only those few.
The numbers below are invented, but the shape is real: a close look is affordable only on a short list.
Worked example: what each pass costs
| Pass | Pages looked at | Time per page | Total time |
|---|---|---|---|
| First pass | 1,000,000 | 0.00002 ms | 20 ms |
| Second pass | 20 | 5 ms | 100 ms |
| Second pass on everything | 1,000,000 | 5 ms | 5,000 seconds |
The cells students fill in are left blank in this preview.
A reranker can be a plain rule
A reranker is whatever scores the short list again. It can be a trained model, a language model or a rule you write. The rule in the Run gives 3 points for a query word in the title and 1 for a word in the body.
A language model reranker reads more closely but costs a call, tokens and time for every query. Start with a rule and measure.
A common mistake
Warning
Many people think a reranker can find pages the first pass missed. It cannot. It only reorders what it is given, so the first pass sets the ceiling. If the answer is not in the top 20, no second pass will show it.
Write the model helper, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const base = process.env.AI_URL || \"http://ai:4000\";' '// The token counts of a reply, or null when the reply carries none.' 'function count(data, name) {' ' return data.usage && typeof data.usage[name] === \"number\" ? data.usage[name] : null;' '}' > model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the model helper, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function ask(messages, sample, max) {' ' try {' ' const res = await fetch(base + \"/v1/chat/completions\", {' ' method: \"POST\",' ' headers: { \"content-type\": \"application/json\", authorization: \"Bearer lab\" },' ' body: JSON.stringify({ model: \"lab\", messages: messages, max_tokens: max || 100 }),' ' signal: AbortSignal.timeout(30000),' ' });' ' if (res.ok === false) throw new Error(\"status \" + res.status);' ' const data = await res.json();' ' const text = data.choices[0].message.content;' ' return { text, inTokens: count(data, \"prompt_tokens\"), outTokens: count(data, \"completion_tokens\"), recorded: false };' ' } catch (error) {' ' console.log(\"Recorded sample: no model answered (\" + error.message + \").\");' ' return { text: sample, inTokens: null, outTokens: null, recorded: true };' ' }' '}' 'module.exports = { ask };' >> model.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the reranker, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const { ask } = require(\"./model\");' 'const query = [\"refund\", \"money\", \"back\"];' 'const found = [' ' { id: \"c1\", title: \"Pay by card or bank transfer\", body: \"Your money leaves your account when you pay.\" },' ' { id: \"c2\", title: \"Refund times\", body: \"Your money is back in five days after we accept a return.\" },' ' { id: \"c3\", title: \"Track a parcel with the order number\", body: \"Type the number to see where it is.\" },' ' { id: \"c4\", title: \"Refund policy and money back\", body: \"Return a parcel within 30 days and ask for a refund.\" },' ' { id: \"c5\", title: \"Change the email address\", body: \"Open your account and edit the address.\" },' '];' > rerank1.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the reranker, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function count(text) {' ' const w = text.toLowerCase().split(\" \");' ' return query.filter(function (q) { return w.includes(q); }).length;' '}' 'function points(c) { return 3 * count(c.title) + count(c.body); }' 'function ids(list) { return list.map(function (c) { return c.id; }).join(\" \"); }' >> rerank1.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the reranker, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'async function main() {' ' console.log(\"first pass: \", ids(found));' ' const second = found.slice().sort(function (a, b) { return points(b) - points(a); });' ' console.log(\"second pass:\", second.map(function (c) { return c.id + \"=\" + points(c); }).join(\" \"));' ' console.log(\"passages read by the second pass:\", found.length);' ' const top = second.slice(0, 3);' ' const lines = top.map(function (c, i) { return (i + 1) + \". \" + c.title; });' ' const prompt = \"Which passage answers: how do I get my money back? Reply with one number only. \" + lines.join(\" \");' ' const reply = await ask([{ role: \"user\", content: prompt }], \"1\", 5);' ' const pick = Number(reply.text.trim().charAt(0));' ' console.log(\"model picks:\", pick >= 1 && pick <= 3 ? top[pick - 1].id : \"no clear pick\");' '}' 'main();' >> rerank1.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run and compare the orders
docker exec -w /home/node/project agent node rerank1.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
When a second pass is not worth it
Measure the hit rate at the top few places before and after. If it barely moves, skip the pass. Skip it too when the first pass is already right, when the list is short, or when the delay would break your latency budget.
The Run is a stand-in: five recorded candidates and a rule. A real system uses a real index and a real model.
One possible output
| Line | What it shows |
|---|---|
| first pass: c1 c2 c3 c4 c5 | The recorded first pass order |
| second pass: c4=9 c2=5 c1=1 c3=0 c5=0 | The rule moves c4 to the top |
| passages read by the second pass: 5 | The cost, counted in passages |
| model picks: c4 | One live pick, or Recorded sample |
The cells students fill in are left blank in this preview.