Search That Understands Meaning

Four components

1. Chunking — splitting documents into retrievable passages
2. Embedding — turning passages into vectors
3. Indexing — storing them so similar meaning can be found
4. Retrieval — finding the most relevant for a query

Teams spend most of their attention on the third and fourth. The first is what actually determines answer quality, and it is usually done with a default setting.

Chunk on meaning

Splitting every thousand characters cuts tables in half, separates headings from their content and produces passages that make no sense in isolation.

• Split on document structure — sections, headings, natural boundaries
• Keep the heading with the content it introduces
• Do not split tables
• Overlap slightly so context is not lost at boundaries
• Carry metadata — source, section, date — with every chunk

Filter before searching

Permissions, recency and document type should filter the candidate set before similarity search, not after. Filtering afterwards is both slower and, for permissions, incorrect.

Most vector stores support metadata filtering during search. Use it rather than post-filtering results.

Combine with keyword search

Pure semantic search fails badly on exact identifiers, which is a common and surprising problem in technical domains. Combining the two handles it.

Keep the index current

Documents change, and a retrieval system that reindexes nightly can be a day out of date on the morning a price changes.

Watch the source and update affected chunks within minutes. Remove deleted documents immediately — superseded content in the index is the main cause of confidently wrong answers.

Frequently asked questions