The problem it solves
Every organisation over about thirty people has knowledge scattered across a document store, a wiki, an email archive and several people's heads. New staff ask questions that are answered somewhere; experienced staff spend time answering them.
Retrieval-based search means someone asks a question in plain language and gets an answer with a citation to the source document, so they can check it. That last part is what makes it trustworthy enough to use.
How it actually works, briefly
- Documents are split into chunks and converted into vectors that capture meaning
- A question is converted the same way and the closest chunks are retrieved
- Those chunks are re-ranked so the best few go forward
- A model answers using only that material, and cites which document it came from
No training on your data is required, updates take effect as soon as a document is indexed, and permissions can be enforced at retrieval time so nobody sees what they should not.
Document quality decides everything
Contradictory policies produce contradictory answers. Three versions of the same handbook produce whichever version was retrieved. Undated documents mean nobody can tell what is current.
Every internal search project turns into a document housekeeping project by week three. Treat that as part of the work rather than as a surprise, and the project succeeds.
- One authoritative version per policy, with older ones archived out of the index
- Dates and owners on documents so currency is visible
- Scanned documents processed to text, or they are invisible to search
- Explicit exclusion of drafts, personal folders and superseded material
Permissions are not optional
The fastest way to create a serious incident is to index HR files and salary data into a system everyone can query. Access control must be enforced at retrieval, filtering by the asking user's permissions before anything is sent to a model.
Design this at the start. Retrofitting per-user filtering onto a system already in use is unpleasant and, in the interim, risky.
Measure with real questions
Collect fifty real questions from staff with agreed correct answers, and use them as your evaluation set. Rerun on every change to chunking, retrieval or prompting.
This is what separates a system that improves from one that is tuned by anecdote and gradually gets worse in ways nobody notices.
Cost and timeline
A production system over a defined document set typically runs £15,000–£40,000 including evaluation, permissions and an interface people will use. Running costs are modest — a penny or two per question at typical volumes.
Six to ten weeks is a realistic timeline, with the first usable version considerably sooner and the remainder spent on retrieval quality and access control.