Documents and databases need different handling
Retrieval works for prose. It does not work for “how many orders over five hundred pounds shipped late last month”, because that answer does not exist in any document.
For those, the model's job is to write the query, and the database's job is to produce the number.
The safe pattern
- The question is translated into a query against a documented schema
- The query is validated — read-only, permitted tables, row limits
- It runs against a reporting replica, never production
- The result is returned as data, and the model explains it
- The query is shown, so the user can check the logic
Showing the query is what makes this trustworthy. A number with no visible derivation is a number nobody should act on.
What to restrict
- Read-only credentials, always
- An allow-list of tables and views, not the whole schema
- Row and time limits, to prevent an accidental full scan
- Permission filters applied in the view, not in the generated query
- No writes, ever, from generated queries
Views are your friend
Expose curated views rather than raw tables: correctly joined, permission-filtered, with sensible column names. Query quality improves dramatically and the risk surface shrinks.
A dozen well-named views usually beats access to two hundred raw tables, for both accuracy and safety.
Where it works and where it does not
| Question type | Fit |
|---|---|
| Counts, sums and simple filters | Strong |
| Trends over time | Strong |
| Joins across two or three tables | Good with views |
| Complex financial calculations | Use a defined report instead |
| Anything a regulator will read | Use a defined report instead |