Handing Over a Machine Learning System to Your In-House Team
Last updated:
The handover that is really a goodbye email
A consultancy builds a pricing model, it performs well, the contract ends. The handover is a repository link, a slide deck and an hour-long call. Six months later the in-house developer needs to retrain it and discovers the training data came from a spreadsheet nobody can find, two features depend on a script that ran on the consultant's laptop, and the thresholds in production do not match anything in the code.
That is not a handover. It is a departure. Machine learning systems are harder to hand over than ordinary software because a lot of their behaviour lives outside the code, in data, decisions and judgement calls made during the build.
What your team must be able to do on their own
The test of a good handover is practical. Before the external team leaves, your people should have done each of these themselves. Watching someone else do it does not count.
- Rebuild the current production model from a clean checkout and get the same results
- Run a retrain on fresh data, evaluate it against the live model and decide whether to promote it
- Read the monitoring and explain whether the model is healthy
- Roll back to the previous version
- Trace a single past prediction back to its inputs and model version
- Follow the incident runbook for a simulated failure
- Make a small change, such as adding a feature, and release it through the normal checks
If any of those cannot be done yet, the handover is not finished, whatever the contract end date says.
The handover pack
| Item | What it contains | Why it matters |
|---|---|---|
| Reproducible training | Code, pinned environment, data extraction queries, fixed cut-off dates | Without it, nobody can safely change the model |
| Model card | Purpose, data used, performance by segment, known limits, intended use | Stops the model being used for things it was not built for |
| Architecture notes | Data flow, serving pattern, dependencies, credentials locations | The map for anyone new |
| Runbooks | Retraining, rollback, incident response, common data faults | Turns emergencies into checklists |
| Monitoring guide | What each chart and alert means and what to do about it | Otherwise alerts get muted |
| Decision log | Why features, thresholds and model types were chosen or rejected | Prevents repeating failed experiments |
| Test suite | Data, code, behavioural and quality-gate tests | Lets the team change things with confidence |
The decision log is the item most often missing and the one in-house teams value most after a year. Knowing that a promising-looking feature was tried and leaked future information saves weeks.
Knowledge that does not fit in documents
Some things only transfer by working together. Why a particular customer segment is excluded from training. Which source system goes wrong every quarter end. How the sales team actually uses the scores, as opposed to how the specification said they would.
- Pair the in-house engineer with the builder on the last retrain before handover
- Have the in-house team lead a release while the external team watches
- Run a simulated incident with the in-house team at the controls
- Introduce the in-house team to the business users of the predictions directly
More general advice on this sits in handovers that do not lose knowledge, which applies well beyond machine learning.
Skills your team needs to own a model
You do not need a research scientist to maintain a production model. You do need someone comfortable with Python and SQL, the specific libraries used, basic statistics for reading evaluation results, and the deployment tooling. Just as important is someone on the business side who owns the model's outcome and reads the reports.
Be realistic about gaps. If the model is a complex deep learning system and your team are strong web developers, plan training time or retain some specialist support for the harder tasks such as rebuilding features. Handing over a system your team cannot yet maintain helps nobody.
A model your team is afraid to retrain is a model that will be switched off within a year.
A sensible handover timeline
- From the start: build with handover in mind, in your repositories and cloud accounts, with documentation written as you go
- Four to six weeks before the end: in-house team joins daily work and shadows releases
- Two to four weeks before: in-house team leads retraining and releases with support
- Handover week: sign off the checklist of things the team has done alone
- After handover: a short period of on-call support for questions, then a check-in after the first retrain
The first point matters more than it looks. Systems built in a supplier's accounts with a supplier's tools are painful to move. Insist on your own accounts and repositories from day one.
How we do it
SpiderHunts plans handover from the first week of a machine learning project: the code lives in the client's repositories, the model card and decision log grow as we go, and the in-house team joins before the end rather than after. We also say plainly when a client's team is not yet in a position to own a system alone, and suggest a staged approach instead.
If you have in-house developers already, our approach to working alongside your in-house team describes how that collaboration works day to day.
Frequently asked questions
What should a machine learning handover include?
What is a model card?
How long does a machine learning handover take?
Can a web development team maintain a machine learning model?
Planning to bring your model in-house?
Tell us who will own it and what they already know. We will outline a handover that leaves your team able to retrain, monitor and fix the model without calling anyone.