One number on an invoice
The AI features went live and people like them: a support assistant, document summaries, a search that understands questions. The provider's invoice arrives each month as a single number, and it has been climbing. The dashboard shows usage by day and by model, perhaps by API key. It does not tell you that most of the spend comes from one feature, or one customer, or one prompt that quietly got longer.
Finance asks whether the AI features pay for themselves. Product asks whether to include them in the cheaper plan. Nobody can answer with numbers.
Why AI spend is hard to see
- Several features share one API key, so the provider cannot separate them.
- Costs depend on tokens, not requests, and token counts vary with prompt length, retrieved context and conversation history.
- Background jobs, retries and agent loops make calls nobody sees in the interface.
- Prompt changes that add instructions or context raise the cost of every call from that point on.
- Different models have very different prices, and it is not always clear which one each feature uses.
The underlying issue is that AI calls are treated like any other API call, when each one carries a variable price. Without recording the context of each call, the bill cannot be explained after the fact.
What not knowing costs
| Problem | Consequence |
|---|---|
| No cost per feature | You cannot tell which features earn their keep |
| No cost per customer | Heavy users on flat plans can make accounts unprofitable |
| No cost per prompt version | A prompt change that doubles context goes unnoticed |
| No alerts | A runaway loop or abuse is found on the invoice |
| Guesswork pricing | Plans are priced without knowing what they cost to serve |
The runaway case is the one that hurts suddenly. An agent stuck in a retry loop or a scraper hitting your chatbot can generate a large bill before anyone looks.
How we make AI costs visible
- Route every AI call through one place. A small internal layer (or a gateway such as LiteLLM or a provider's own tooling) handles calls to OpenAI, Anthropic Claude, Azure OpenAI or other models.
- Record context with every call: feature, customer or account, user, model, prompt version, input and output tokens, cached tokens, latency and calculated cost.
- Send that data to a store you can query, alongside your other application logs, or to an LLM observability tool such as Langfuse or Helicone where that suits you.
- Build dashboards for the questions people actually ask: cost per feature, per customer, per plan and per prompt version over time.
- Set alerts and limits: spend thresholds per day, per customer and per feature, and caps on agent loops and retries.
- Use the data to reduce cost where it makes sense: trimming context, caching repeated answers, using smaller models for simple steps, and batching work that does not need an instant reply.
We keep personal data out of these logs where it is not needed, storing token counts and metadata rather than full prompts unless you have decided otherwise and have a reason to.
With costs traced
You can see which feature and which customers drive spend, and how that changes when a prompt or model changes. Pricing decisions are based on the real cost to serve. An unusual spike raises an alert on the day it starts. And cost reductions are aimed at the calls that matter instead of general belt-tightening.
Does this match your situation?
- Your AI provider bill has grown and you cannot say why.
- Several features or products share one API key.
- You do not know what an average customer costs in AI usage.
- Nobody would be alerted if AI spend jumped overnight.
- You are deciding how to price AI features without cost data.