One answer in a long sheet
Chunking means splitting a document into pieces small enough to be found and read one at a time. It matters because retrieval returns whole pieces, so the size of a piece decides what you get back.
A library has one information sheet with opening hours, borrowing rules and study rooms. A visitor asks about fines. The answer is one sentence. Handing over the whole sheet buries it.
By the end of this lesson
Note
I can split a document so that each piece can be found.
Key words
| Word | What it means | Example |
|---|---|---|
| Chunk | one piece of a split document | |
| Chunk size | how many words one chunk holds | 8 |
| Overlap | the words two chunks in a row share | |
| Heading | a title line that starts a section | |
| Retrieval | finding the chunks that fit a question |
The cells students fill in are left blank in this preview.
Why a document is cut into chunks
One vector for a whole sheet blurs every topic on it into one average. A question about fines then looks no closer to the sheet than a question about rooms.
A small chunk is about one thing, so its vector points somewhere clear. It is also short enough to put in a prompt later, next to a few others.
Chunk size and overlap
The simplest split counts words. A chunk of 8 words holds 8 words. The next chunk starts a few words earlier than where the last one ended. Those shared words are the overlap.
Overlap stops an idea being cut in half. A sentence at the edge of one chunk then also sits at the start of the next.
Worked example: ten words, size 4, overlap 1
| Chunk | Starts at word | Words |
|---|---|---|
| 1 | 1 | a b c d |
| 2 | 4 | d e f g |
| 3 | 7 | g h i j |
The cells students fill in are left blank in this preview.
Splitting at headings keeps an idea together
A word count does not know where an idea ends. A heading does. If the document has headings, start a new chunk at each one.
The script below does both. It splits the library sheet at its headings, then cuts one section by words with an overlap.
Write the chunking script, part 1
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'const sheet = \"# Opening hours\nThe library opens at 9 am from Monday to Friday. On Saturday it opens at 10 am. It is closed on Sunday.\n# Borrowing\nYou can borrow 8 books for 3 weeks. A late book costs 20p a day. You can renew a book twice online.\n# Study rooms\nA study room holds 6 people. Book at the desk or online. A booking lasts 2 hours.\";' 'function byHeading(text) {' ' const parts = [];' ' const lines = text.split(\"\n\");' ' for (let i = 0; i < lines.length; i++) {' ' if (lines[i].startsWith(\"# \")) {' ' parts.push({ heading: lines[i].slice(2), text: \"\" });' ' } else {' ' const last = parts[parts.length - 1];' ' last.text = (last.text + \" \" + lines[i]).trim();' ' }' ' }' ' return parts;' '}' > chunk.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Write the chunking script, part 2
docker exec -w /home/node/project agent sh -c "printf '%s\n' 'function byWords(text, size, overlap) {' ' const words = text.split(\" \");' ' const chunks = [];' ' let start = 0;' ' while (start < words.length) {' ' chunks.push(words.slice(start, start + size).join(\" \"));' ' if (start + size >= words.length) break;' ' start = start + size - overlap;' ' }' ' return chunks;' '}' 'const parts = byHeading(sheet);' 'console.log(\"By heading:\", parts.length, \"chunks\");' 'parts.forEach(function (part) { console.log(\"-\", part.heading + \":\", part.text.split(\" \").length, \"words\"); });' 'const pieces = byWords(parts[1].text, 8, 3);' 'console.log(\"By words, size 8, overlap 3:\", pieces.length, \"chunks\");' 'pieces.forEach(function (piece) { console.log(\"-\", piece); });' >> chunk.js"Shown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
Run it and read the chunks
docker exec -w /home/node/project agent node chunk.jsShown here, not run. Create a free account to press Run and type it into your own lab’s terminal.
What it prints
| Line | What it shows |
|---|---|
| By heading: 3 chunks | One chunk for each heading |
| - Borrowing: 22 words | The size of the middle chunk |
| By words, size 8, overlap 3: 4 chunks | The borrowing text cut into four pieces |
| - You can renew a book twice online. | Its first three words also end the chunk before it |
The cells students fill in are left blank in this preview.
A common mistake
Warning
Many people think bigger chunks are safer because they hold more. A big chunk has one vector that blurs several topics, and it fills the prompt. Pick the smallest chunk that still holds a whole answer.