Perfect is not the requirement
Every business has messy data and waiting for it to be clean means waiting forever. The requirement is knowing where the mess is, so it can be handled deliberately.
An integration built on data whose problems are known is fine. One built on data assumed to be clean is not.
The three that matter
- Consistent identifiers — one customer is one record, not four
- Current master data — the price list and product catalogue are actually right
- Accessible history — you can get at past records in a usable form
Duplicate customer records are the single most common blocker. Validation against a master with four versions of the same customer produces four different answers.
How to check quickly
- Count distinct customers, then count them by name similarity — compare
- Pick twenty products and verify the price against reality
- Try exporting a year of history and see what happens
- Ask whoever maintains the data what they do not trust
That last question usually gets you the whole answer in ten minutes.
Fix what blocks, tolerate the rest
| Problem | Action |
|---|---|
| Duplicate master records | Fix before building |
| Stale prices or catalogue | Fix before building |
| Inconsistent free-text notes | Tolerate — that is what the model is for |
| Missing historical fields | Tolerate, exclude from validation |
| No export capability | Resolve, or the project cannot proceed |
Deduplication is itself an AI job
Matching records that refer to the same entity but are written differently is exactly what models do well, and it is a safe batch job with a review queue.
It is frequently a good first project precisely because it clears the ground for everything after it.