Chunking strategies compared: what actually moves precision
Leela Desai
Founding Engineer
We ran three chunking strategies across four corpora — banking policies, API documentation, support tickets, and long-form research reports — and measured retrieval precision, answer faithfulness, and cost. Here's what actually moved the numbers.
The contenders
- Fixed-window: hard token windows with overlap; the baseline
- Recursive: split on paragraphs, fall back to sentences, then words
- Semantic: embed sentences, break where similarity drops
Results
| Strategy | Context precision | Faithfulness | Index size | Ingest cost | | --- | --- | --- | --- | --- | | Fixed 512/64 | 0.81 | 0.90 | 1.0× | 1.0× | | Recursive 512/64 | 0.87 | 0.94 | 0.97× | 1.02× | | Semantic | 0.86 | 0.93 | 0.91× | 2.4× |
Recursive chunking won or tied everywhere. Semantic chunking matched it on quality but cost 2.4× to ingest — every boundary decision needs sentence embeddings — and the gap never justified itself on our corpora.
Where fixed-window falls down
Fixed windows split mid-sentence at exactly the wrong moments. In the banking corpus, the sentence "…a monthly maintenance fee of $5 applies until the balance is restored" landed across a boundary — and the question "what's the fee for going under the minimum?" retrieved neither half confidently. Overlap papers over this, but recursive splitting simply doesn't create the wound.
Overlap: the 10–15% rule
Zero overlap saves index space and reliably costs you boundary answers. Past ~20%, you're embedding the same sentences twice for no measurable precision gain. Every corpus we tested optimized inside 10–15%.
Practical recipe
- Start recursive, 512 tokens, 64 overlap
- Build a 50-question eval set from real user queries before tuning
- Sweep size in 256512768 — pick by context precision, not vibes
- Revisit only when your document mix changes
Chunking is upstream of everything: no reranker can rescue a chunk that cut the answer in half.