What You Are Actually Paying For in an AI Integration

Where the money goes

Every request carries the instruction, the retrieved context, the conversation history and the question. Output is charged too, usually at a higher rate.

Most expensive integrations are expensive because they send far more context than the task needs, not because the model is costly.

Five levers

1. Retrieve fewer, better passages — five relevant beats twenty adequate
2. Summarise long history rather than resending every turn
3. Cache repeated instructions and stable context where the provider supports it
4. Route by difficulty so simple steps use cheaper models
5. Cap output length, because generation is the expensive direction

Improving retrieval usually cuts cost and improves accuracy at the same time. It is the rare optimisation with no trade-off.

Long conversations get expensive quickly

Resending the full history every turn means cost grows with the square of the conversation length. A twenty-turn conversation can cost more than the first nineteen combined.

Summarise older turns and keep the recent ones verbatim. Users notice nothing; the bill halves.

Model the cost before you build

Those four give a monthly figure within about twenty per cent, which is enough to decide.

Caps are not optional

A hard daily cap, a per-user cap and an alert well below both. A loop or an abusive user should cost you an alert, not an invoice.

Decide the behaviour at the cap too — degrade or queue rather than crash.

Frequently asked questions