The average is not the experience
A model reported as 88% accurate is 88% accurate across whatever mix the test set happened to contain. Individual groups may be far better or far worse, and the customers in the worse groups experience the model as unreliable.
This matters commercially as well as ethically. If the group the model handles worst happens to be your highest-value customers, the headline figure is actively misleading about the model's business value.
Which slices to check
- Commercially important groups - largest customers, highest-margin products, strategic accounts
- Data-poor groups - new customers, recently added products, small branches
- Structurally different groups - a different channel, a different country, a different contract type
- Protected characteristics, where the decision affects people - required in some contexts and good practice in most
- Time - performance in the most recent period against earlier ones
The data-poor category is usually where the worst performance hides, for an obvious reason: the model had least to learn from. It is also frequently where growth is.
Doing it without fooling yourself
Slicing many ways means many chances to find an apparently poor group by chance. Small slices produce noisy estimates, and a group of forty cases can look alarming for no reason.
- Set a minimum slice size below which you report uncertainty rather than a point estimate.
- Show confidence intervals, not bare percentages, so small groups visibly carry wide ranges.
- Decide the slices before looking, so you are testing rather than hunting.
- Repeat the check on a later period before acting - a genuine weakness persists.
What to do about a weak subgroup
| Cause | Response |
|---|---|
| Too few training examples | Collect more, or weight the group up in training |
| Genuinely different behaviour | Consider a separate model for that group |
| Missing features for that group | Find features that exist for them |
| Inherently harder to predict | Set a different threshold, or exclude and handle manually |
That last row is legitimate and under-used. A model that abstains for a group it handles badly, routing those cases to a person, is more honest than one that produces unreliable scores everywhere.
Put it in the documentation
Subgroup performance belongs in whatever documentation accompanies the model, alongside the headline figure. Anyone deciding whether to rely on it needs to know where it is weak.
It also protects the project. A known, documented weakness that someone chose to accept is a managed risk; the same weakness discovered later by a customer is an incident.
An average accuracy figure is a promise to nobody in particular.