The short answer
A small corpus hides retrieval problems because almost anything relevant is found. As content grows, near-duplicates and loosely related material crowd out the right answer.
Measure retrieval on its own, because a good final answer can mask poor retrieval right up until it does not.
What degrades as content grows
| Problem | Effect |
|---|---|
| Near-duplicate documents | Results fill with variations of one thing |
| Outdated versions still indexed | Confident answers from superseded content |
| Chunks too small | Context lost, meaning changes |
| Chunks too large | Relevant part diluted |
| No filtering | Searching everything, every time |
Outdated content is the one that causes real damage. An answer drawn from a superseded policy is worse than no answer, because it is believed.
Filter before you search
- Narrow by what you already know: tenant, product, date range, permission.
- Search within that subset rather than across everything.
- Keep structured attributes alongside the text for filtering.
- Exclude superseded or archived content by default.
- Make the filter part of the query, not a post-processing step.
Point five matters for correctness as well as speed. Filtering afterwards can leave you with nothing when everything retrieved was excluded.
Keep the index current
- Reindex on change rather than on a slow schedule
- Remove deleted content promptly
- Track when each item was last indexed
- Detect drift between the source and the index
- Have a way to rebuild from scratch, tested
A stale index produces answers that were right last quarter, and nothing in the output indicates that.
Measure retrieval separately
Keep a set of questions with the documents that should be retrieved for each, and check that they are, independently of what the final answer says.
That separation tells you which half to fix. Without it, a wrong answer could be retrieval, generation or both, and you are guessing.