Find the five closest of five thousand
A vector store keeps chunks with their vectors and finds the ones nearest to a question. It matters because retrieval is only useful if it is fast and returns the best few, not all of them.
A college library has five thousand chunks of guides and rules. A student types a question. The program must put the question next to each chunk and bring back the closest three.
By the end of this lesson
Note
I can store vectors and find the nearest ones.
Key words
| Word | What it means | Example |
|---|---|---|
| Vector store | a list of chunks and their vectors | |
| Query | the question, turned into a vector | |
| Score | how close a chunk is to the query | 0.82 |
| Top k | the k chunks with the best scores | k = 3 |
| Index | a structure that skips most of the list |
The cells students fill in are left blank in this preview.
A vector store is a scored list
The simplest store is a list. Each item holds an id and a vector. To search, turn the question into a vector, score every item with the cosine, sort the scores and keep the first k.
That is all "nearest neighbour search" means. The best result is the item with the highest score.
Worked example: query [1, 0] against three items
| Item | Vector | Cosine | Rank |
|---|---|---|---|
| a | [1, 0] | 1.00 | 1 |
| b | [1, 1] | 0.71 | 2 |
| c | [0, 1] | 0.00 | 3 |
The cells students fill in are left blank in this preview.
A count vector only matches the same words
There is no embedding model in this lab, so the next script makes its own vectors. It counts how many times each word of a fixed list appears in a chunk. It also trims a trailing "s", so "opens" and "open" match.
This is a stand-in. A real embedding also matches "borrow" with "lend". A count vector cannot, because it only sees the same words.
Write the store script, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const vocabulary = [\"open\", \"close\", \"saturday\", \"sunday\", \"borrow\", \"book\", \"late\", \"fine\", \"room\", \"library\", \"friday\", \"week\"];' 'const chunks = [' ' { id: \"hours-week\", text: \"The library opens at 9 am and closes at 6 pm from Monday to Friday.\" },' ' { id: \"hours-weekend\", text: \"On Saturday the library opens at 10 am and closes at 2 pm. It is closed on Sunday.\" },' ' { id: \"borrowing\", text: \"You can borrow 8 books for 3 weeks. You can renew a book twice online.\" },' ' { id: \"fines\", text: \"A late book costs 20p a day. The fine stops at 10 pounds.\" },' ' { id: \"rooms\", text: \"A study room holds 6 people. Book a room at the desk or online for 2 hours.\" },' '];' 'function stem(word) { return word.length > 3 && word.endsWith(\"s\") ? word.slice(0, -1) : word; }' > store.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the store script, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function vectorOf(text) {' ' const words = text.toLowerCase().replace(/[^a-z0-9 ]/g, \"\").split(\" \").map(stem);' ' return vocabulary.map(function (term) {' ' return words.filter(function (word) { return word === term; }).length;' ' });' '}' 'function cosine(a, b) {' ' let dot = 0, sa = 0, sb = 0;' ' for (let i = 0; i < a.length; i++) { dot += a[i] * b[i]; sa += a[i] * a[i]; sb += b[i] * b[i]; }' ' return sa === 0 || sb === 0 ? 0 : dot / Math.sqrt(sa * sb);' '}' >> store.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the store script, part 3
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const store = chunks.map(function (chunk) {' ' return { id: chunk.id, vector: vectorOf(chunk.text) };' '});' 'function search(question, k) {' ' const query = vectorOf(question);' ' return store' ' .map(function (item) { return { id: item.id, score: cosine(query, item.vector) }; })' ' .sort(function (a, b) { return b.score - a.score; })' ' .slice(0, k);' '}' 'const question = \"when does the library open on saturday\";' 'console.log(\"Question:\", question);' 'search(question, 3).forEach(function (hit) {' ' console.log(hit.score.toFixed(2), hit.id);' '});' >> store.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run it and read the scores
docker exec -w /home/node/project agent node store.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What it prints
| Line | What it shows |
|---|---|
| Question: when does the library open on saturday | The query |
| 0.77 hours-weekend | The best chunk: it names Saturday |
| 0.58 hours-week | The next: it has open and library |
| 0.00 borrowing | The third, with no word in common |
The cells students fill in are left blank in this preview.
Why an index exists
Scoring every item is fine for five chunks, or five thousand. It is slow for fifty million, because every search must do all that work.
An index groups close vectors so a search can skip most of the list. Most indexes are approximate: they are much faster and may now and then miss the true nearest one.
A common mistake
Warning
Many people think the top result always holds the answer. It is only the nearest chunk. If nothing in the store is close, the top result is the least bad one. Look at the score before you trust it.