Stop asking which model is best
The question assumes a single answer for a workload that is actually several tasks with different requirements. Most business features involve classification, extraction, retrieval and generation, and those want different models.
The productive question is: for each step, what is the cheapest model that meets the quality bar?
Route by difficulty
| Task | Model tier | Why |
|---|---|---|
| Classification, routing | Small and fast | Simple decisions, high volume |
| Extraction from documents | Small to mid | Structured output, verifiable |
| Answering from retrieved context | Mid | Needs comprehension, not deep reasoning |
| Drafting for human review | Mid to large | Quality is visible to the reader |
| Multi-step reasoning, agents | Large | Planning quality determines the outcome |
A confidence threshold that escalates uncertain cases to a stronger model gives most of the quality at a fraction of the cost.
Build an evaluation set before choosing
The only way to choose responsibly is to test candidates on your own task with your own data. Public benchmarks measure general capability and correlate loosely with performance on your specific job.
A few hundred real examples with agreed correct answers is enough. That set is also what lets you switch models later in an afternoon rather than a month, which matters more than the initial choice.
Design for replacement
Models improve and prices fall on a timescale of months. Anything hard-coded to one provider's specifics becomes a migration project when a better option appears.
- Keep prompts in configuration, not scattered through code
- Abstract the provider behind a thin interface
- Avoid depending on provider-specific quirks unless the benefit is large
- Re-run the evaluation set on new releases as a routine task
Latency is a product decision
A user waiting for a response has a different tolerance from a background job. Interactive features need sub-second first tokens and benefit from streaming; batch processing can use slower, cheaper, stronger models.
Decide the latency budget per feature before choosing the model, because it eliminates options quickly and prevents a late redesign.
Consider where it runs
For most business features, hosted APIs from the major providers are the sensible default: better models, no infrastructure, and enterprise terms that address most data concerns.
Self-hosted open models make sense with strict data residency requirements, sustained high volume, or a need for offline operation. The trade is capability and engineering time against control.