Filter before, never after
The rule is simple and frequently broken: decide what this user may see, retrieve only from that, then generate.
If restricted content reaches the model, it will influence the answer even when the specific sentence is removed afterwards. There is no reliable way to unring that bell.
Use the source system's permissions
Do not build a second permission model that has to be kept in step with the first. Read the user's actual entitlements from your identity system or the document store and apply those.
A parallel model drifts within months, and the drift is invisible until something leaks.
What needs attention
- Documents inheriting permissions from folders that have since moved
- Passages carrying permission metadata that is not refreshed on change
- Shared conversation history exposing another user's retrieved content
- Logs containing retrieved passages, readable by administrators
- Cached answers served to a user with different entitlements
Caching is a permission problem
Answer caching improves speed and cost and it must be keyed by permission scope, not only by question text. Two users asking the same question may be entitled to different answers.
This is one of the easiest mistakes to make and one of the most damaging.
Test it deliberately
- Create a low-privilege test account
- Ask direct questions about restricted material
- Ask indirect questions that would require it to answer well
- Check the logs for what was retrieved, not only what was returned
That fourth step is the one that finds the real problems.