The question assumes a single answer
“Which model is best” presumes one workload. Most business applications involve several tasks — classification, extraction, retrieval, generation — and those want different models.
The productive question is: for each step, what is the cheapest model that meets the quality bar for that step?
Routing by difficulty
| Task | Tier | Why |
|---|---|---|
| Classification, routing | Small and fast | Simple decisions, high volume |
| Extraction | Small to mid | Structured output, easily verified |
| Answering from context | Mid | Comprehension rather than deep reasoning |
| Drafting for review | Mid to large | Quality is visible to the reader |
| Multi-step reasoning | Large | Planning quality determines the outcome |
Test on your data, not on benchmarks
Public benchmarks measure general capability and correlate loosely with performance on your specific job. A few hundred of your own cases with agreed answers settles the question in an afternoon.
That set is also what lets you switch models later without re-testing everything by hand, which matters because better and cheaper options appear every few months.
Design for replacement
- Prompts in configuration, not scattered through code
- The provider abstracted behind a thin interface
- No dependence on provider-specific quirks unless the benefit is large
- The evaluation set rerun automatically on any model change
Latency is a product decision
A user waiting has a different tolerance from a background job. Interactive features need fast first tokens and benefit from streaming; batch processing can use slower, cheaper, stronger models.
Decide the latency budget per feature before choosing, because it eliminates options quickly and prevents a late redesign.