Data Quality: The Five Problems Worth Fixing First
Last updated:
Not all bad data is expensive
Data quality projects go wrong when they aim for clean data as an end in itself. There is always more mess, and much of it costs nothing. The useful question is which specific defects cause money to leave the business.
That reframing usually shrinks the project from a year to a quarter and makes it far easier to justify.
The five that reliably cost
- Duplicate records. Two versions of the same customer means duplicated marketing spend, split history, and the embarrassment of two account managers calling the same person.
- Stale contact details. Bounced email harms deliverability for everyone else, and undeliverable post costs postage plus the lost contact.
- Inconsistent identifiers between systems, which makes every cross-system report a manual reconciliation.
- Missing values in decision fields. A blank you must chase before acting delays revenue and consumes a person.
- Silent format drift — dates as text, numbers with stray characters, mixed units. Quietly corrupts every calculation downstream.
Put a number on each before you start
| Problem | How to quantify |
|---|---|
| Duplicates | Duplicate rate × contacts per year × cost per contact |
| Stale contacts | Bounce rate × volume × cost per send, plus lost opportunity |
| Inconsistent IDs | Hours per month reconciling × loaded rate |
| Missing fields | Chasing time per record × volume, plus delay cost |
| Format drift | Time spent correcting reports, plus any decision made on wrong figures |
Some will come out at a few hundred pounds a year. Leave those alone. The exercise is as valuable for what it lets you skip as for what it prioritises.
Fix at the point of entry, not in the warehouse
Cleaning downstream is endless because the source keeps producing mess. Validation at entry — format rules, required fields, duplicate detection before saving, dropdowns instead of free text — stops the problem being created.
One good validation rule at the point of entry is worth a year of cleaning scripts, because it prevents rather than corrects. It is also much less popular, since it makes someone's data entry slightly slower today to save everyone time later.
Deduplication needs a decision, not just an algorithm
Matching is the easy part. The hard part is deciding what happens when two records disagree: which address wins, whose history is kept, who is told. Get those rules agreed by the business before writing any code.
And keep a reversible audit trail. Merges that turn out to be wrong are common, and being unable to undo one destroys trust in the whole exercise.
Then keep it clean
A one-off clean-up decays within a year. Add a monthly quality report — duplicate rate, completeness on key fields, bounce rate — with an owner and a threshold that triggers action.
This is unglamorous and it is the difference between a data quality project and a data quality practice.
Frequently asked questions
How do we find duplicates when the data is messy?
Should we clean historical data or start fresh?
What does a data quality project cost?
Who should own data quality?
Suspect your data is costing you but cannot prove it?
We start by quantifying, not cleaning. Tell us which reports you distrust and we will tell you what the mess is worth in pounds.