The short answer
Anything billed by usage needs a ceiling you control, not just the provider's. Set limits per user, per period and per request, and alert on spend as it accumulates rather than discovering it on the invoice.
The expensive incidents are rarely malicious. They are a retry loop, an automated client, or a feature used far more than expected.
Where the surprises come from
- A retry loop with no cap, hitting the provider repeatedly
- An automated client or scraper using an unauthenticated endpoint
- A single user with a use case nobody anticipated
- Inputs far longer than expected, since cost scales with size
- A bug that calls the provider inside a loop
- A feature becoming popular faster than the budget allowed
The second is why unauthenticated endpoints that cost money per call are a bad idea. Anything that costs you money should require identification.
The limits to set
| Limit | Protects against |
|---|---|
| Per request input size | One enormous input |
| Per user per day | One user consuming everything |
| Per account per month | Overall budget |
| Concurrent requests | Bursts and retry storms |
| Retries per request | Loops |
All five are straightforward to implement and each one has prevented a real incident somewhere. Setting them before launch costs an hour.
Alert on spend, not just errors
- Track spend continuously rather than at billing time.
- Alert at a fraction of the monthly budget, early enough to act.
- Alert on rate of spend, not only on total.
- Attribute spend per feature so you know what is consuming it.
- Review monthly against what the feature is worth.
Point three catches the incident while it is happening. A total-based alert fires after most of the damage.
Decide what happens at the limit
Degrade rather than break where you can: a cheaper model, a cached answer, a simpler non-AI path, or an honest message. Silently failing is the worst option.
And make sure hitting a limit is visible to you as well as to the user, because a limit quietly throttling a legitimate use case is its own problem.