Precision vs Recall: The Business Trade-off Behind Every Model
Last updated:
Two ways to be wrong, one dial
Every model that flags things, whether fraud, churn risk, faulty parts or urgent emails, can make two kinds of mistake. It can flag something that turns out to be fine. Or it can let through something it should have caught. You cannot eliminate both. You can choose which one you would rather live with.
Precision and recall are the names for those two sides. They sound technical and they are not. They are the most business-relevant numbers in the whole of machine learning, and the people best placed to balance them are usually not the data scientists.
The definitions, with a fishing net
Imagine casting a net for a particular fish.
- Recall is the share of the target fish in the lake that ended up in your net. High recall means you caught nearly all of them.
- Precision is the share of what is in your net that is the target fish. High precision means you did not drag up much else.
A huge net with tiny holes gets excellent recall and poor precision: every target fish, plus boots, weed and everything else. A small, carefully aimed net gets excellent precision and poor recall: almost nothing wrong in it, and most of the fish still swimming.
Translated: a fraud model with high recall catches most fraud but blocks lots of honest customers. One with high precision rarely blocks the innocent but lets more fraud through.
A worked example with numbers
Say an insurer receives 10,000 claims a month and around 200 of them are fraudulent. A model scores every claim, and the insurer picks a threshold above which claims go to the investigations team. Here is how three settings might play out, as an illustration:
| Threshold | Claims flagged | Fraud caught | Precision | Recall |
|---|---|---|---|---|
| Strict | 100 | 80 | 80% | 40% |
| Middle | 400 | 150 | 38% | 75% |
| Loose | 1,500 | 190 | 13% | 95% |
None of these is correct in the abstract. If the investigations team can handle 400 claims a month and each fraud costs 8,000 pounds, the middle setting looks attractive. If the team can handle 100 and every investigation of an honest claimant risks a complaint, the strict setting wins. If regulators take a very dim view of missed fraud, the loose setting might be required, with more investigators hired to match.
How to choose, using money and capacity
- Price a false alarm. Staff time to check it, customer friction, any refunds or goodwill.
- Price a miss. The loss, the downstream clean-up, the reputational or regulatory exposure.
- Check capacity. How many flagged cases can a person actually review per day.
- Plot value at each threshold. Money saved from caught cases minus the cost of false alarms, within capacity.
- Pick the peak, then sanity-check it with the people who will handle the flagged cases.
This takes an afternoon with a spreadsheet once the model produces scores. It is the most valuable afternoon in most projects, and it is routinely skipped in favour of a default threshold of 50% that nobody chose.
Typical leanings by use case
| Use case | Usually favour | Why |
|---|---|---|
| Medical screening | Recall | A miss can be very serious; follow-up tests catch false alarms |
| Spam filtering | Precision | Losing a real customer email is worse than seeing some spam |
| Churn outreach | Recall, within team capacity | A friendly call to a happy customer costs little |
| Automatic account suspension | Precision | Wrongly locking out a customer is costly and visible |
| Predictive maintenance | Recall | Unplanned downtime usually costs far more than an inspection |
These are leanings, not rules. Your costs decide.
Two thresholds are often better than one
A practical trick: instead of one cut-off, use two. Cases above a high threshold get acted on automatically, cases in the middle band go to a human, and cases below the low threshold pass. You get high precision on the automated part and high recall overall, with human attention focused where the model is least sure.
This is how we design most classification systems at SpiderHunts, whether it is a fraud check or an email router built as part of an automation project. The width of the middle band becomes a lever the business controls: widen it when stakes rise, narrow it as trust in the model grows.
Who should own the dial
The threshold is a policy decision dressed up as a technical setting. It should be owned by whoever owns the outcome, such as the head of fraud, the customer success lead or the operations manager, and it should be easy for them to change without a developer.
If the only person who can move the threshold is the data scientist, the business has handed a policy decision to the one person who does not carry its consequences.
Revisit it when costs change. A new investigator, a spike in fraud, a change in refund policy: all of these move the right balance. For more on judging whether a model is good enough in the first place, see how accurate a model needs to be and our wider notes on evaluating AI features.
Frequently asked questions
What is the F1 score?
Can a model have high precision and high recall at the same time?
Is precision the same as accuracy?
How often should we review the threshold?
Trying to decide where your model's threshold should sit?
Bring the costs of a false alarm and a miss, and a sample of predictions. We will help you find the setting that makes the most sense for the business.