Chunking

Text Chunking for RAG

Why Chunking Matters
RAG systems need to split documents into smaller pieces for embedding and retrieval. Chunk size and strategy significantly impact retrieval quality.

Chunking Strategies

Fixed Size
Split by character/token count:

def fixed_chunk(text: str, chunk_size: int = 500, overlap: int = 50) -> list:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

Semantic Chunking
Split at natural boundaries:
- Paragraphs
- Sections (headers)
- Sentences
- Topics (using embeddings)

Recursive Splitting
Try multiple separators hierarchically:

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=500,
    chunk_overlap=50,
    separators=["

", "
", ". ", " ", ""]
)
chunks = splitter.split_text(document)

Chunk Size Guidelines
| Use Case | Recommended Size | Notes |
|----------|------------------|-------|
| Q&A retrieval | 100-500 tokens | Precise answers |
| Summarization | 500-1000 tokens | Coherent context |
| Code | Function-level | Logical units |
| Tables | Full table | Preserve structure |

Overlap Considerations
| Overlap % | Benefit | Tradeoff |
|-----------|---------|----------|
| 0% | Storage efficient | May split mid-concept |
| 10-20% | Balanced | Standard choice |
| 30-50% | Context preservation | More storage, redundancy |

Document-Specific Chunking

Code

def chunk_code(code: str) -> list:
    # Split by function/class definitions
    # Keep docstrings with their functions
    # Respect indentation boundaries

Markdown

def chunk_markdown(md: str) -> list:
    # Split at headers
    # Keep header hierarchy metadata
    # Preserve code blocks intact

Tables
Keep tables together:

def handle_table(table_text: str) -> list:
    # Never split a table
    # Include table caption
    # Add column headers to each chunk if splitting rows

Metadata
Attach context to chunks:

chunk = {
    "text": "...",
    "source": "document.pdf",
    "page": 5,
    "section": "Introduction",
    "char_start": 1500,
    "char_end": 2000
}

Metadata enables filtering, citation, and context reconstruction.