Chunking#

Chunking mediates the fundamental RAG tradeoff: small chunks retrieve precisely but fragment context; large chunks preserve context but dilute embeddings.

Strategies#

| Strategy | How it splits | Best for | | --- | --- | --- | | Recursive | Paragraphs → sentences → words, respecting boundaries | Most corpora (default) | | Fixed-window | Hard character/token windows with overlap | Uniform, structure-poor text | | Semantic | Embedding-similarity breakpoints | Long-form prose, reports |

The strategy is a config value (CHUNKING_STRATEGY) — changing it re-chunks on the next ingestion without code changes.

Size and overlap#

Start at 512 tokens with 10–15% overlap and tune against your eval set:

• Overlap exists so answers spanning a chunk boundary survive retrieval — without it, the sentence that answers the question can be split in half
• Below ~128 tokens, chunks lose the context needed to be understood alone
• Above ~1024 tokens, embeddings average over topics and retrieval precision drops measurably

Try the interactive chunk visualizer to see how size and overlap reshape a real document.

Metadata#

Every chunk carries provenance: source file, page, position, and heading path. This is what makes citations exact rather than approximate.