Three genuine reasons to self-host
- Data cannot leave your infrastructure. A contractual or regulatory constraint, not a preference.
- Sustained high volume, where GPU capacity running continuously is cheaper than per-token pricing.
- Offline or air-gapped operation, where there is no external connectivity at all.
If none of those applies, hosted APIs are almost always the better economic and operational choice.
The cost that is not the model
Open weights are free. Running them is not. GPU instances are billed by the hour whether busy or idle, which means the economics depend on utilisation rather than on request count.
The crossover is about sustained load. A model serving a few thousand requests a day is usually cheaper on a hosted API; one serving continuous high volume can be cheaper self-hosted, if you have the engineering to keep it running.
The engineering you are taking on
- Deployment, scaling and failover for GPU workloads
- Model updates and the evaluation that follows each one
- Monitoring for both availability and output quality
- Capacity planning, since demand spikes cannot be absorbed elastically as cheaply
- Security of the infrastructure holding your data
That is a real ongoing commitment, and it is the reason many self-hosting projects quietly move back to hosted APIs after a year.
Capability, honestly
Open models have improved markedly and, for many business tasks — classification, extraction, summarisation, straightforward drafting — they are entirely adequate.
At the frontier of reasoning and complex instruction-following, the leading proprietary models generally remain ahead. Whether that gap matters depends entirely on your task, which is why an evaluation on your own data settles it rather than a benchmark table.
A pragmatic middle path
Use hosted APIs with regional processing and enterprise terms for most work, and self-host only the specific workload where the constraint genuinely applies.
Design the system so the provider is abstracted, and you can move a workload either way without rebuilding around it.