Making Your Documents Answerable

The idea, in one paragraph

Instead of asking a model what it knows, you search your own documents for passages relevant to the question, hand those passages to the model, and ask it to answer from them alone.

That is the whole mechanism. Everything else — embeddings, vector databases, chunking strategies — is implementation detail in service of one goal: find the right paragraph, then answer only from it.

What it is genuinely good for

• Internal knowledge: policies, procedures, how a thing is done here
• Product and technical documentation, especially large sets
• Contract and policy questions where the source must be citable
• Historical records — past tickets, past projects, past decisions
• Regulatory material where an answer must point at its source

The common thread is that a correct answer exists in writing somewhere and finding it is the hard part. Where no correct answer exists in your documents, retrieval cannot conjure one and should say so.

The preparation that decides everything

Retrieval quality is dominated by document quality. We have watched more of these projects succeed or fail on the content than on the engineering.

1. Currency. Out-of-date documents produce confidently out-of-date answers. Retire superseded versions rather than leaving both.
2. Contradictions. If two documents disagree, the system will cite one of them at random. Resolve them first.
3. Structure. Clear headings help enormously, because they let a passage carry its own context.
4. Self-contained passages. A paragraph beginning “this does not apply if…” is dangerous in isolation.
5. Scanned images. These need OCR, and the OCR quality becomes your ceiling.

Every retrieval project we have run has spent more time on content than on code, and every one that skipped that work had to come back and do it.

Citations are not optional

Every answer should carry a link to the source passage. It lets the user verify, it makes wrongness diagnosable, and it changes the tool's social meaning: it becomes a search engine that explains itself rather than an oracle.

In practice users click citations far more than anyone expects, especially early on. That is trust being built, and it is worth the engineering.

What it costs to build and run

For a well-organised document set, four to eight weeks to build. The ongoing cost has three parts: the model calls, the storage and search, and — the one people forget — keeping the content current.

• Re-indexing when documents change, ideally automatically
• A review of low-confidence and unanswered questions, weekly at first
• Removing superseded material, which nobody wants to own
• Periodic re-evaluation against a saved question set

When it is the wrong approach

If the answer requires combining information from many documents at once — “how many contracts expire next quarter” — retrieval is the wrong tool. That is a database query, and the right answer is to extract structured data first and query it.

It is also wrong where the answer must be exact and current: stock levels, prices, account balances. Those come from systems, not documents, and the model should be given the number rather than asked to find it.

Frequently asked questions