RAG chunking and source boundaries
Chunk contents
A chunk is a retrievable part of a source, not a new independent authority. Preserve its document identity and revision along with its text. Overlap can keep a sentence’s context near a boundary, but duplicated text should not become two independent supporting sources.
Split a fictional maintenance note into bounded pieces. Keep the final short piece rather than discarding it. An empty document should produce no content chunks. Store stable identifiers so a citation can be checked against the exact retrieved piece after indexing.
Local limits
The retrieval guide, checked November 14, 2026, describes hosted chunking and indexing options. This exercise implements a small local splitter. Its chosen size is a task requirement, not a universal best chunk size or a measured recommendation for every corpus.
Continue with fixture vectors and the carried handbook index project. Evaluate a live corpus before choosing production chunk boundaries.