Why keyword search fails internally
Internal documents are written by many people over years with no shared vocabulary. The policy is called one thing by HR and another by operations; the procedure everyone needs is titled something nobody would search for.
Keyword search requires guessing the author's words. Semantic search matches on meaning, so a question phrased in the searcher's own terms finds the document regardless of its wording.
How it works, briefly
Documents are split into passages, each converted to a numeric representation capturing its meaning. A query is converted the same way, and the system returns passages whose representations are closest.
That is why it tolerates different vocabulary: two passages about the same thing sit near each other regardless of wording. It is also why it sometimes returns something topically related but not the answer - closeness is not the same as correctness.
The problems that matter in practice
| Problem | Consequence | Approach |
|---|---|---|
| Permissions | Someone finds a document they should not see | Filter by the searcher's access before returning |
| Stale documents | Superseded policy returned as current | Index status and date; prefer current versions |
| Near-duplicates | Five versions of one document returned | Deduplicate; mark the authoritative copy |
| No good answer exists | Something loosely related is returned | Show confidence; allow 'nothing found' |
Permissions is the one that must be right before launch. A search system that ignores access controls is a data breach with a search box, and retro-fitting filtering is much harder than building it in.
Answers on top of search
Adding a language model that reads retrieved passages and composes an answer is attractive and adds risk. The answer reads authoritatively whether or not the retrieved passages supported it.
- Always cite the source passages, and make them one click away
- Have it say when the documents do not contain the answer, rather than composing one
- Prefer quoting to paraphrasing for anything procedural or regulatory
- Log questions with no good answer - that list is a map of your documentation gaps
That last point often becomes the most valuable output. Knowing what people repeatedly ask and cannot find tells you exactly what to write next.
Keeping it current
An index built once and never refreshed becomes wrong quietly. Documents change, get superseded, get deleted, and the index keeps serving what it captured.
Incremental updates on a schedule, plus removal of deleted documents, are essential rather than optional. The failure is invisible - the system keeps returning results, they are simply out of date, which is worse than returning nothing.
A search tool that confidently returns last year's policy is worse than one that finds nothing.