Chunking
Splitting documents into smaller passages before embedding them for retrieval. How you chunk strongly affects the quality of a RAG system.
Before documents can be retrieved in a RAG system, they are split into smaller passages, or chunks, each of which is turned into an embedding. Chunking sounds trivial but is one of the most consequential choices in a pipeline.
Chunks that are too large dilute the embedding and pull in irrelevant text; too small and they lose the context needed to make sense. Good strategies respect the document’s structure, such as splitting on headings or paragraphs, and often overlap chunks slightly so an answer is not cut in half at a boundary.