How We Run a Machine Learning Discovery Workshop

Most ML projects go wrong before anyone trains a model

When a machine learning project fails, the post-mortem rarely blames the algorithm. It blames a vague goal, data that could not support the goal, or a prediction that nobody was ever going to act on. All three were knowable in the first week. The discovery workshop exists to find them while finding them is still free.

So we do not open with modelling. We open with the business decision, the people who make it and the data they make it with. If those three line up, the modelling is usually the least risky part of the project. If they do not, no amount of clever modelling will rescue it.

Who should be in the room

The single most useful attendee is the person who makes the decision the model is meant to improve, every day, by hand. Not their manager, although the manager is welcome. The credit controller who decides which accounts to chase. The planner who decides how much stock to reorder. They know which cases are easy, which are hard and which rules of thumb they already use, and that knowledge shapes the model more than any feature engineering we do later.

• The decision-maker: the person who does the job the prediction supports
• The budget owner: someone who can say whether the value justifies the build
• Whoever knows the data: often an analyst, sometimes the person who built the spreadsheet in 2019
• From our side: the engineer who would lead the project, plus our commercial contact

We do not send a salesperson to the workshop and a different engineer to the build. The person asking about your data is the person who would be working with it.

The agenda, roughly in order

The session usually runs two to three hours, remote or in person, and follows the same shape each time so nothing important gets skipped.

Pinning down the decision, not the model

The most useful sentence we can write in a workshop has three parts: we predict X, so that person Y does Z differently, which is worth roughly W. “Predict which customers will churn” is not that sentence. “Flag accounts likely to cancel within 60 days so the account team calls them first, where saving one account is worth about £4,000” is.

Getting to that sentence often changes the project. A demand forecast becomes a reorder recommendation for the 200 lines that cause most stockouts. A lead-scoring model becomes a simple ranking of the morning's enquiries. The narrower version is almost always cheaper, faster to prove and more likely to be used.

A prediction that does not change what somebody does on Monday morning is a chart, and charts are cheaper to build than models.

Looking at the data live

We ask you to share a screen and open the real data, not a cleaned sample prepared for the meeting. In twenty minutes of scrolling we can usually answer the questions that decide feasibility: is the outcome recorded, how far back does history go, how many examples of the rare case exist, and did the definition of anything change halfway through.

This is where many workshops turn. A churn project discovers that cancellations were only logged properly from last spring. A forecasting project finds three years of sales but no record of stockouts, so the history shows what was sold rather than what was wanted. Neither kills the project, but both change the plan and the price, and it is far better to learn them here than in week four.

If the data cannot be shown live for confidentiality reasons, a pseudonymised export sent in advance works almost as well. We will sign your NDA before either.

What you leave with

Within a few working days of the session you receive a short written document. It is deliberately plain, and it is yours to keep whether or not you work with us.

• The decision statement, in the three-part form above, written in your words
• What the data can and cannot support, with the specific gaps named
• A recommended first step, usually a short proof of value on your real data
• A price for that first step and a range for the full build
• The assumptions behind both, and what changes if any of them are wrong
• The risks we would watch, in order of how much they could cost

For most enquiries the workshop and the written scope are free. Where the process is genuinely complex, with several decisions or data sources that need mapping before anyone can quote, we will say so up front and propose a short paid discovery instead. You will know which it is before the session is booked.

When the workshop ends with “do not build a model”

A fair number of workshops end with us recommending something other than machine learning. A validation rule that stops the bad data at source. A report that shows the planner the three numbers they actually use. A feature already in your CRM that nobody switched on. We say so in the room, because a good ML partner should tell you when not to use ML, and because the honest version of this is how most of our referrals arrive.

If the answer is machine learning, the proof of value is normally the next step. If you want to understand how we think about the wider partnership first, why SpiderHunts for machine learning covers our five-stage approach, and our machine learning development service lists the kinds of models we build.

Frequently asked questions