Getting Staff Buy-In for Machine Learning: Why People Override the Model

The model works. Nobody uses it.

An illustrative story we have seen in several versions. A 200-person distributor commissions a demand forecasting model. On the test data it beats the buyers' own forecasts comfortably. It goes live with a dashboard. Three months later, the buyers are still ordering from their spreadsheets. The model's numbers sit on a screen, consulted occasionally and then adjusted until they match what the buyer was going to order anyway.

The general advice on change management is sound, and we have written about getting teams to adopt AI before. But machine learning predictions have a particular problem that deserves its own treatment. The people being asked to use them are usually experienced, often good at the job, and being asked to defer to a number they cannot interrogate.

Overrides are not the enemy. They are data. The goal is not maximum compliance with the model; it is the best decisions, which sometimes means the human was right.

The real reasons people override

1. They cannot see why. A score of 0.73 with no explanation asks for trust without giving a reason. Experienced staff, reasonably, do not give it.
2. They saw it fail on a case that mattered. One confident, wrong prediction on a big customer is remembered far longer than a hundred correct ones on small orders.
3. They know something the model does not. A customer called to say they are opening a new site. A competitor has a stock problem. That knowledge is real and the model has no way to receive it.
4. The risk is personal. If a buyer follows the model and runs out of stock, it is their failure. If they follow their instinct and it goes wrong, it is ordinary bad luck. Deferring to the model feels riskier to the individual even when it is safer for the business.
5. It threatens their role. Rarely said out loud. A model that forecasts better than a buyer raises an obvious question about the buyer.
6. It is inconvenient. The prediction lives in a separate screen, arrives too late, or in a format that does not match how they work.

Nobody overrides a model they understand, trust and find convenient. Work out which of the three is missing.

Measure overrides before trying to fix them

Most businesses do not know their override rate, let alone whether overrides help. Set up simple tracking first:

• Record the model's prediction and the final human decision for every case
• Capture a short reason for significant overrides, from a pick list plus optional free text
• After the outcome is known, compare: was the model or the human closer?
• Break results down by person, product category, customer segment and size of override

The results usually surprise both sides. Typically some overrides are consistently valuable, such as adjustments for promotions the model does not know about, and some are consistently harmful, such as rounding up orders out of caution. Some staff improve on the model and some do not. That breakdown turns an argument about trust into a conversation about evidence.

Design changes that earn trust

Ranges deserve a special mention. 'Forecast 420 units' invites an argument. 'Likely between 380 and 460, with 420 most likely' invites a judgement, and experienced staff are far more comfortable adjusting within a range the model has said is plausible.

Involve the sceptics early

The most effective single step is to involve the people who will use the predictions in building them. Not a focus group at the end, but a senior buyer, underwriter or planner in the room while features are chosen and results reviewed.

They will name data the modellers did not know existed. They will point to cases where the historical outcome was misleading. They will tell you which errors are tolerable and which are unforgivable. And when the model goes live, there is someone respected on the team who can say, truthfully, that they helped shape it.

This is standard practice in how SpiderHunts runs data science projects. It costs a few hours a week of an expert's time, and it is repaid many times in adoption.

When mandating the model is right, and when it is not

Sometimes, after measurement, it is clear that overrides make results worse. In that case it is reasonable to change the process: follow the model by default, allow overrides within a set tolerance, and require a recorded reason and sign-off beyond it. Done openly, with the evidence shared, most teams accept this.

What does not work is mandating the model before that evidence exists. People comply in the letter, find workarounds in practice, and the business loses both the model's value and the staff's judgement.

And sometimes the measurement shows the humans are right more often. Then the model needs improving, narrowing to the cases where it is strong, or retiring. We would rather tell a client that than keep a model in production that the experts are correctly ignoring.

A practical sequence for a stalled model

1. Start logging predictions, decisions and outcomes if you are not already
2. Interview a handful of the heaviest overriders about why
3. After enough outcomes, analyse which overrides helped and which hurt
4. Fix the top one or two causes from the table above
5. Share the analysis with the team, including where they beat the model
6. Agree default behaviour and override bounds based on the evidence
7. Review the override rate and value every quarter

If you want broader background on how models like this are built and evaluated, our machine learning guide is a good starting point.

Frequently asked questions