The Monthly Cost of Keeping an AI App Running

Model usage is the smallest line

For a typical business workload — extraction, classification, retrieval-based answers — model cost is fractions of a penny to a few pence per completed task. At ten thousand tasks a month that is tens to low hundreds of pounds.

Clients consistently expect this to dominate. It rarely does.

Where the money actually goes

• Human review — frequently the largest line in the first quarter, falling as thresholds tune
• Retrieval infrastructure — embedding, storage and re-embedding when documents change
• Evaluation — rerunning the test set on every change, plus refreshing it periodically
• Monitoring for quality drift, which conventional software does not need
• Maintenance at 20–30% of build annually

How we keep model cost down

1. Route by difficulty — a small model for classification, a stronger one only where reasoning is needed
2. Send less context — most retrieval sends more chunks than accuracy requires
3. Cache the stable parts of prompts, which is close to free money at volume
4. Cap and alert — hard spend limits per day and per user

Routing by difficulty commonly halves an AI bill with no measurable quality difference. It has to be designed in; retrofitting it means re-evaluating everything.

Quality drifts even when nothing changes

Your documents change, your customers change, the model provider updates something. A system that was 94% accurate in March can be materially worse in September with no code change.

That is why evaluation is a running cost rather than a one-off. Automated reruns on a schedule catch it before users do.

Modelling it before you build

Cost per completed task at your real volume, plus infrastructure, plus the review time you expect in months one to three, plus maintenance. Then model it again at ten times the volume.

Usage pricing has a habit of surprising people at the second order of magnitude, and it is much better discovered on a spreadsheet.

Frequently asked questions