What is Chunking?

Embedding models have input limits, and retrieving a whole document when one paragraph is relevant wastes context. Chunking splits documents into retrievable units — the trade being that too-small chunks lose context and too-large ones dilute the embedding.

Chunking on structural boundaries — sections, paragraphs — generally beats fixed token counts, because it respects the document's own units of meaning.