AI Integration
Choosing How Often Your AI Integration Runs
Last updated:
Default to batch
Real-time processing is more expensive to build, more expensive to run and considerably harder to make reliable. It is right when someone is waiting and wrong otherwise.
A great deal of engineering has been spent making things instant that nobody looks at until the next morning.
Where real time is genuinely needed
- A visitor waiting for an answer on your website
- An agent with a customer on the phone
- An enquiry where response speed decides the sale
- A safety-relevant classification
What batch gives you
| Benefit | Why |
|---|---|
| Lower cost | Batching amortises overhead and enables cheaper tiers |
| Simpler retries | A failed run reruns; nobody is waiting |
| Easier rate limit handling | Work spreads across the window |
| Better review ergonomics | A queue to work through beats constant interruption |
| Simpler monitoring | One run to check, not a continuous stream |
The middle ground
Frequent small batches — every five or fifteen minutes — feel near-instant to most business processes while keeping all the operational advantages of batch.
That is the right answer more often than either extreme, and it is what we build by default.
Match the schedule to the work
- Enquiry handling: every few minutes, or real time if it wins business
- Document processing: hourly, or on arrival for urgent types
- Reporting and summaries: daily, before people need them
- Enrichment and research: overnight, when capacity is cheap
Frequently asked questions
Does batch feel slow to users?
Not at five or fifteen minute intervals for most internal processes. Ask what the actual expectation is rather than assuming instant.
Is real time much more expensive?
Often two to three times to build, and somewhat more to run. Worth it where it wins business; wasteful where it does not.
Can we start batch and add real time?
Yes, and it is the sensible order. The pipeline is largely the same; the trigger and the interface change.
What about overnight runs failing?
Alert on completion as well as failure, so a run that never started is noticed. A silent skipped run is the classic batch failure.
Being quoted for real-time processing?
Ask whether anyone is actually waiting. Frequently batch does the job for a fraction of the cost.