Chunk Overlap

Chunk overlap prevents important context from being split at chunk boundaries. Problem: Fixed-size chunking can split sentences, paragraphs, or logical units, making retrieved chunks incomplete. Solution: Overlap consecutive chunks by N tokens, ensuring boundary content appears in at least one complete chunk. Typical values: 10-20% overlap (50-100 tokens for 500-token chunks). Too little: context splits remain; too much: redundancy and increased storage. Example: 400-token chunks with 50-token overlap → each boundary region covered in two chunks. Trade-offs: Increased storage (overlap creates redundancy), more chunks in index, potential for duplicate retrieval results. Deduplication: Remove near-duplicate chunks from retrieval results, or prefer higher-ranked version. Alternatives to overlap: Semantic chunking at natural boundaries, sliding window retrieval (compute on-the-fly), parent-child retrieval. Best practices: Match overlap to typical semantic unit sizes in your documents, monitor for retrieval duplicates, combine with sentence-aware splitting when possible. Simple but effective technique for improving RAG context quality.