What a Machine Learning Proof of Value Looks Like

Proof of concept is the wrong question

A proof of concept asks whether something is technically possible. For most business machine learning the answer is obviously yes. Of course a model can predict late payments, or demand, or which enquiries convert. The question that decides whether you should spend money is different: can it predict them well enough, on your data, to change a decision in a way that is worth more than the build and the upkeep?

That is what our proof of value is designed to answer. It is short, it is priced as a fixed piece of work, and it is allowed to end with a no. A proof of value that cannot fail is a sales exercise.

How it differs from a proof of concept and a pilot

We skip the first column almost entirely. The second is where the important uncertainty is resolved, and the third comes later, as part of the build.

Step one: agree what success means before we start

Before we write any modelling code we agree, in writing, the threshold that would justify a build. It is expressed in business terms and tied to the decision, not to an abstract accuracy score.

• “Among the 50 accounts flagged each week, at least half go on to pay late”
• “Forecast error on the top 300 lines is lower than the current reorder spreadsheet by a clear margin”
• “Routing the top-scored third of enquiries first would have captured most of last quarter's closed deals”

Writing the threshold down first matters more than it sounds. Once results arrive, everyone is tempted to decide that whatever number came out is good enough. Agreeing it beforehand removes that temptation from both sides, including ours.

Step two: the baseline nobody finds exciting

The first thing we build is deliberately simple: the rule your team already uses, written as code, or a plain statistical model such as a logistic regression. Every more sophisticated model has to beat it on the same held-out data.

Sometimes the baseline is enough. If a three-line rule captures most of the value, we will tell you, and the cheapest good outcome of a proof of value is discovering that you did not need machine learning at all. Our post on machine learning versus a rules engine goes further into when that happens.

If a model cannot clearly beat the rule your best person already uses, it is not a model you need. It is a model you would have to maintain.

Step three: a real model on real history

Only then do we build the candidate model, usually gradient boosting on tabular data, and test it the way it would actually be used. That means training on older history and testing on the most recent period, never shuffling time together, because a model that has quietly seen the future always looks brilliant in a notebook.

We also look at where it is wrong. Averages hide the cases that matter. A model can be accurate overall and useless for your largest customers, or strong in one region and poor in another. The decision-maker from your side reviews a sample of predictions with us, because they will spot a result that is technically correct and commercially absurd within seconds.

What you receive at the end

1. An evaluation report comparing the baseline and the model against the agreed threshold, in plain language
2. The evaluation set itself, so any future model, ours or anyone else's, can be measured the same way
3. A list of what drove the predictions, so you can judge whether the model's reasoning makes business sense
4. The data problems we found, with an estimate of what fixing each would be worth
5. A recommendation: go, go with a narrower scope, or stop
6. A fixed price for the production build if the recommendation is go

The code and the evaluation set are yours, like everything else we produce. If you take them to another team, they start from a measured position rather than from a claim.

How long it takes, what it costs and when it is the wrong step

A typical proof of value runs two to three weeks once we have the data. It is priced as a separate, fixed piece of work, quoted in the scope that follows our discovery workshop. The largest variable is how quickly data access arrives, not how long the modelling takes.

It is the wrong step when the value is so obvious and the risk so low that measuring it would cost more than just building carefully, or when the data needed does not exist yet. In the first case we go straight to a build with an evaluation gate inside it. In the second, the right project is collecting the data properly for six months, and we will help you design that instead. Either way, the machine learning service you end up with starts from evidence.

Frequently asked questions