Making Sure the Bill Is Predictable

Context, not volume

Most expensive AI services are expensive because they send far more context than the task needs, not because they handle many requests.

Sending an entire document when three paragraphs would do multiplies your bill and slows every response. Better retrieval is a cost lever, not only an accuracy one.

Five levers

1. Retrieve fewer, better passages
2. Summarise conversation history rather than resending everything
3. Cache stable instructions and repeated context where the provider supports it
4. Route by difficulty — cheap models for simple steps
5. Cap output length, because generation costs more than input

Track cost per request, not total

Total cost rising with volume is expected. Cost per request rising means something has changed — longer prompts, more retries, or retrieval returning more than it should.

• Record tokens in and out on every request
• Aggregate by endpoint and by caller
• Alert when the per-request figure moves
• Review monthly against the previous month

Hard caps with decided behaviour

A cap with no decided behaviour becomes a crash at the worst moment. Decide what happens before you need it.

Conversations get expensive quickly

Resending full history each turn means cost grows with the square of conversation length. A twenty-turn conversation can cost more than the first nineteen combined.

Summarise older turns, keep recent ones verbatim. Users notice nothing and the cost halves.

Frequently asked questions