What Changes in Your Business After Machine Learning Goes Live

Launch is the middle, not the end

The project plan ends with go-live. The value starts there. A model that predicts late deliveries, scores leads or categorises invoices changes how people spend their day, what managers look at and what happens when things go wrong. None of that is in the model code.

We have seen technically excellent models switched off within six months, not because they stopped working, but because the business around them never adjusted. Here is what actually changes, and what to prepare.

Jobs shift from doing to checking

The most visible change is in the work of the people who used to make the decision. Take an illustrative accounts team coding 3,000 supplier invoices a month. Before, they coded every one. After, the model codes most of them confidently and routes perhaps 15% to a person because it is unsure or the invoice is unusual.

• The work becomes harder per item, because the easy cases have gone and only exceptions remain
• People need clearer rules for exceptions, which were previously handled by instinct
• Productivity measures based on items processed stop making sense
• Some staff enjoy the change; others miss the rhythm of routine work, and that is worth acknowledging

Plan training for the exception work before launch, and involve the team in setting the confidence threshold. They know which mistakes hurt.

Someone has to own the model

Every live model needs a named owner. Not necessarily a data scientist: often a business-side person who reviews a monthly report, backed by a technical person who can retrain and fix it.

If nobody's name goes in those boxes, the model has a shelf life, and nobody will know when it expires.

Monitoring becomes a routine

Models degrade because the world changes: new products, new customers, a supplier changing their invoice layout. The degradation is usually gradual and invisible without measurement.

1. Track input data for sudden changes, such as a field that goes blank or a new category appearing
2. Compare predictions with actual outcomes as soon as outcomes are known
3. Watch the rate at which people override the model; a rising override rate is an early warning
4. Alert when any of these moves beyond an agreed range

Our guide to automated data quality checks covers the input side, which catches a surprising share of problems before they reach the model.

Decisions and accountability change

When a model is involved in a decision, managers ask different questions. Why was this customer flagged? Why did the forecast miss? Was the mistake the model's, the threshold's or the person's who overrode it?

It helps to decide in advance. Record the prediction, the confidence, the final decision and who made it. Agree that people may override the model and should say briefly why. Those override reasons turn out to be the best source of improvements for the next version.

Where decisions affect individuals, such as credit, hiring or insurance, obligations under data protection law and the EU AI Act's requirements for certain high-risk uses make explanation and human oversight a legal matter rather than a nice-to-have. Check this before launch, not after the first complaint.

Costs and data habits change too

A live model brings running costs: hosting, data pipelines, monitoring tools and the owners' time. In our experience it is sensible to plan for ongoing costs of a meaningful fraction of the original build each year. If that surprises the finance team in month four, the model's future becomes a budget argument.

There is also a quieter change. Once a model depends on data, people start caring about data quality in a way they did not before. The warehouse team records reasons for delays more carefully because the forecast uses them. That improvement is a genuine side benefit, and worth encouraging explicitly.

A pre-launch checklist

• Named business owner and technical owner, with the responsibilities written down
• Exception process documented and staff trained on it
• Performance measured against actual outcomes, with a review date in the diary
• Override logging and a simple way to record the reason
• Alerts on data, performance and cost
• A retraining plan and a budget line for ongoing running costs
• A plan for what happens if the model is switched off

When SpiderHunts takes a machine learning model into production, this checklist is part of the handover. If your model is still on its way there, it is worth reading alongside the mistakes businesses make on their first ML project, several of which only become visible after launch.

Frequently asked questions