Accuracy is one of several costs
Model selection is often treated as a search for the highest score. In production, accuracy trades against cost per prediction, latency, explainability, retraining time and how many people can maintain it.
A model a few points better that costs considerably more to run, cannot explain itself and takes a day to retrain is not obviously the better choice. Whether it is depends on what the extra accuracy is worth in your specific decision.
Convert accuracy into money before deciding
The question is not whether the larger model is better but whether the difference changes enough decisions to justify the cost.
Illustrative reasoning: if a 2% accuracy improvement changes the outcome on a small share of cases, and each changed case is worth a modest amount, the annual benefit may be less than the additional running cost. That arithmetic is worth doing explicitly rather than assuming better is better.
| Consideration | Favours smaller | Favours larger |
|---|---|---|
| High prediction volume | Yes - cost scales | |
| Tight latency budget | Yes | |
| Explanation required | Usually | |
| Frequent retraining needed | Yes - faster cycles | |
| Small team maintaining it | Yes | |
| Accuracy directly drives large sums | Yes | |
| Complex unstructured data | Often yes |
Tabular business data rarely needs the big options
For the data most businesses hold - orders, customers, transactions, jobs - gradient boosted trees remain extremely competitive. They train in minutes, run cheaply, handle missing values, and support per-prediction explanation readily.
Deep learning earns its additional cost mainly with images, audio, and large volumes of text. Applying it to a spreadsheet-shaped problem usually adds cost without adding accuracy.
The costs that appear later
- Retraining time - a model taking a day to train gets retrained less often, so it drifts further
- Debugging - understanding why a complex model produced an odd answer takes longer
- Hiring and handover - fewer people can maintain an unusual architecture
- Dependency weight - large frameworks bring more to keep patched
- Infrastructure lock-in - models needing specific hardware constrain where they run
These are invisible at selection time and dominate the total cost of ownership over a few years.
Start small, and be prepared to stop there
The sensible sequence is to establish a simple baseline, measure it properly, and only add complexity where the measured gain justifies the cost.
Frequently the simple version is good enough, and the honest conclusion is to stop. That is a successful outcome rather than a compromise, and it leaves you with something you can actually maintain.
The best model is the simplest one that changes the decision correctly.