Missing is not random in business systems
Statistical texts distinguish data missing at random from data missing for a reason. Business data is almost always the second. A blank field usually means a process did not happen.
- No delivery date because the order was never despatched
- No phone number because the customer refused to give one
- No credit score because the check was skipped for a small order
- No inspection result because the item passed and only failures are recorded
- No discount because the sale went through at list price
In each case the blank carries information. Replacing it with an average destroys that information and can invert the meaning entirely - imputing an average credit score for customers who were never checked treats them as typical when they were systematically different.
Add a flag before you fill anything
The most useful single technique is also the simplest: for any field with meaningful missingness, add a was_missing indicator alongside it. Fill the value however you like, and let the model use the flag to learn what the blank meant.
This costs one column and preserves information that every imputation method otherwise discards. Where the missingness is predictive, the flag frequently turns out to be among the more useful features.
Choosing a fill when you need one
| Approach | Reasonable when | Risk |
|---|---|---|
| Leave as missing | The model handles it natively | Not all do |
| Zero or a sentinel | Missing genuinely means none | Confused with a real zero if not flagged |
| Median or mode | Missing is incidental and rare | Shrinks variance, hides the pattern |
| Predict from other fields | Fields are strongly related | Leakage risk; adds a model to maintain |
| Carry last value forward | Time series where state persists | Invents stability that may not exist |
Several modern algorithms handle missing values directly and often better than imputation. That is usually the first thing to try before building anything elaborate.
The leakage trap in imputation
Imputing with a statistic calculated across the whole dataset leaks information from test into training, and from the future into the past. The fill must be calculated on training data only and applied to everything else.
For time series this is stricter still: the fill for a given date must use only data available before it. This is a common source of backtests that cannot be reproduced live, as covered in our note on backtesting a forecast.
When too much is missing
Sometimes the honest answer is that a field cannot be used. If a column is 80% blank and the missingness correlates with the outcome, no imputation strategy rescues it - anything you fill in is largely invention.
Better to drop it, note why, and if it matters, fix the process that leaves it blank. A field properly captured from now on becomes usable within a year, which is often faster than it sounds.
Imputing a value does not create information. It only hides that you had none.