Machine Learning Projects We Turn Down

A model that should not exist costs more than the invoice

Turning down paid work is uncomfortable for any agency. It is more important in machine learning than in most software, because a bad ML project fails slowly. The model gets built, the scores look reasonable, and it takes months before anyone admits nobody is using the predictions or that they were never reliable. By then the budget, the goodwill and the team's appetite for data projects have all gone.

So we look for the warning signs in the first conversation and the discovery workshop. The situations below cover most of the machine learning enquiries we decline, or more often redirect into something smaller that will work.

1. The prediction would not change a decision

The commonest one. Someone would like to know which customers are likely to churn, but there is no retention team, no budget for offers and no process for acting on a list. The model would be accurate and irrelevant.

We ask what somebody would do differently on receiving the prediction, and what that is worth. If nobody can answer, we suggest fixing that first. Sometimes the answer turns out to be a report, and the model can come later once the action exists.

The question is never whether we can predict it. It is who does what differently when we do.

2. The outcome was never recorded

Supervised machine learning learns from past examples where the answer is known. If your system records orders but not why they were cancelled, or leads but not which ones became customers, there is nothing to learn from. No algorithm can recover an outcome that was never written down.

Here we usually recommend starting to capture the outcome properly, often a single required field, and coming back in six months. It feels like a delay. It is actually the fastest route to a model that works, and we will help design the capture so the data is usable when it arrives.

3. The volume is too low to pay for it

A model that improves a decision made twenty times a year rarely earns back its build and upkeep, however clever it is. The same applies when the rare event you want to predict has only happened a few dozen times in your history, because there is not enough to learn from or to test against.

4. A rule would do the job

Some problems look like machine learning and are really a missing validation rule. Orders flagged as risky turn out to be the ones with a mismatched postcode. Late payers turn out to be the accounts over a credit limit that nobody enforces. When the pattern fits in a sentence, writing it as a rule is cheaper, instant to change and completely explainable.

We check for this during the proof of value by building the rule as the baseline. If it captures most of the value, we recommend the rule. It is the answer that earns us least and saves you most, and our post on when a rule beats a model explains how to spot it yourself.

5. The model would decide about people with nobody checking

We do not build models that make consequential decisions about individuals without human review: rejecting job applicants, refusing credit, withdrawing a service. Those decisions need explanation, a route to challenge and a person accountable for them, and in the UK and EU they increasingly carry legal obligations as well, including the EU AI Act requirements phasing in for high-risk uses.

What we will build is the same model as a recommendation, with the reasons shown and a person making the call. That version is usually more useful anyway, because it keeps the human judgement that catches the cases the history never covered.

• Fully automated rejection of applicants or customers: we decline
• A ranked shortlist with reasons, reviewed by a person: we build
• Automatic credit refusals with no explanation: we decline
• Risk scores that route cases to a reviewer: we build

A few other reasons that come up

• The decision has already been made. The model is wanted to justify a conclusion, and a result that disagrees would be ignored.
• The target keeps moving. The process being predicted is about to be redesigned, so a model trained on the old one is wasted.
• Data access is impossible. The data sits with a third party who will not provide it, and no workaround exists.
• The accuracy bar is absolute. A wrong answer is unacceptable and there is no review step to catch it.

What a no from us looks like

A short written note within a couple of working days: what we understood, why we think a model is the wrong step now, and what we would do in your position. Often that includes a specific rule, report or data-capture change, and occasionally the name of an off-the-shelf product that already does the job.

A no is not always permanent. Sometimes the right sequence is six months of recording the right outcome, followed by a model that finally has something to learn from. If you want to see what a good fit looks like, our machine learning service page lists the problems we take on, and when we turn a software project down covers the same principle for non-ML work.

Frequently asked questions