All terms
Glossary
Chunking
Splitting documents into sections that can be searched individually and handed to a model.
Sections that are too large dilute retrieval, because a hit drags in much that does not fit. Too small and context is torn apart, leaving the answer without grounding.
Better approaches split along document structure, headings and paragraphs, rather than by fixed character count.
How you notice it
- Answers contain fragments without context.
- Documents are long and inconsistently structured.
- Search finds the right file but the wrong passage.
Frequently asked
How large should a chunk be?
As large as one thought, usually 300 to 800 words with some overlap. The boundary matters more than the number: split at headings and paragraphs, never mid sentence. And every chunk carries its document title and section, otherwise the model lacks context.