Work That Happens Without Anyone Asking
Last updated:
Silent failure is the norm
A scheduled task that stops running produces no error anywhere. The report is not sent, the sync does not happen, and nobody notices until someone asks why a number looks wrong.
Alert on completion, not just on failure. A job that never started looks identical to a job with nothing to do, and only a completion alert distinguishes them.
What every scheduled task needs
- Idempotency — running twice must not double the effect
- Overlap protection — a slow run must not be joined by the next one
- Logging of start, finish and what was processed
- A completion signal that is monitored
- A time limit, so a stuck job does not run forever
Use the framework's scheduler
Modern frameworks provide a scheduler defined in code, driven by a single system cron entry. That keeps the schedule in version control and visible to developers.
A dozen separate crontab entries edited on the server is the alternative, and it is invariably out of date and undocumented.
Design for restartability
- Process in batches with progress recorded
- A restarted job should resume, not restart from the beginning
- Failures on one record should not stop the run
- Failed records quarantined with the reason
A nightly job that fails at ninety per cent and has to start again from nothing will eventually fail to complete at all.
Watch the clock
| Issue | Guard |
|---|---|
| Daylight saving changes | Schedule in UTC |
| Jobs overlapping | Locking |
| Jobs running longer over time | Duration monitoring |
| Several heavy jobs at once | Stagger the schedule |
| Month-end volume spikes | Test at realistic volume |
Frequently asked questions
How do we know a scheduled job ran?
What if a job takes longer than its interval?
Should scheduled work go through the queue?
What about time zones?
Overnight job that might not have run?
If you cannot tell without checking manually, that is the problem. Completion alerting is a small fix.
Related services
What we build for problems like this one