Supervised or Unsupervised Learning: Let Your Data Decide

The question that settles it in five minutes

Open the spreadsheet or database export you are thinking of using. Look for a column that contains the thing you want to predict, filled in for past cases. Did this customer churn. Was this claim fraudulent. How many units sold that week. If that column exists and is reasonably complete, you can do supervised learning. If it does not, you cannot, however much data you have.

That sounds brutally simple, and it is. Articles comparing the approaches in the abstract, including our own overview of the three learning types, are useful background. This post is about the decision in front of you: what your data already allows, and what it would cost to get the other kind.

Supervised learning: learning from recorded answers

The word supervised refers to the answers acting like a teacher. The software guesses, checks against the recorded outcome, and adjusts. After enough examples it guesses well on cases it has not seen.

Typical business uses look like this:

• Predicting which leads will convert, using past leads marked won or lost
• Estimating delivery times from past deliveries with actual arrival times
• Flagging invoices likely to be paid late, using payment history
• Categorising expense lines, using thousands already categorised by the finance team
• Forecasting weekly demand from sales history

The strength is that success is measurable. You hold back some past cases, ask the model to predict them, and compare against what really happened. You know roughly how good it is before anyone relies on it.

Unsupervised learning: finding structure nobody labelled

Unsupervised learning is given the data without answers and asked to find shape in it. The two uses we see most in business are grouping and spotting oddities.

Grouping means clustering customers, products or sites by how they behave rather than how you have always categorised them. A subscription box company might discover its 'monthly' customers are really three groups: people who never change anything, people who skip every third box, and people who customise every order. That is useful for marketing and pricing, even though nobody told the software those groups existed.

Spotting oddities means anomaly detection: this supplier invoice looks nothing like the previous 400 from them, this login pattern is unusual, this machine's vibration readings have drifted.

The catch is that there is no answer key. The software will always find groups, even in data where the groups mean nothing. Somebody who knows the business has to look at the output and decide whether it is insight or noise.

Side by side, for the decision you are making

When you want supervised but have no labels

This is the common situation. A business wants to predict fraud, or which support tickets are urgent, but nobody ever recorded the outcome in a usable way. There are four realistic routes, and they have different costs.

1. Start recording now. Add the outcome field to the process and wait. Cheapest in money, most expensive in time. Often the right answer anyway.
2. Label a sample by hand. Have people who know the work mark up a few thousand past cases. A week or two of someone's time can be enough for a first model.
3. Derive labels from other systems. Refunds, chargebacks and cancellations often reveal the outcome indirectly, if you can join the records up.
4. Use unsupervised methods first. Anomaly detection can shortlist suspicious cases for humans to review, and those reviews become labels for a supervised model later.

The fourth route is underrated. It turns an unsupervised project into a labelling machine, and a year later you have the data for the supervised model you wanted in the first place.

Where people go wrong

The most expensive mistake is inconsistent labels. If three people have been tagging tickets as urgent using three private definitions, the model learns the average of their moods. Before training anything we sit with the people who created the labels and ask how they decided. Frequently that conversation is more valuable than the model.

The second is choosing unsupervised learning because it seems easier, then being unable to say whether it worked. Segmentation with no plan to act on the segments produces a nice slide and nothing else.

Clustering will always return clusters. The algorithm cannot tell you whether they matter, only whether they exist in the numbers.

How we approach it

At SpiderHunts we start a data science engagement by auditing what outcomes are actually recorded, how reliably, and since when. That audit usually takes a few days and decides the approach for us. If the labels are good, we go supervised because the results can be measured. If they are not, we design the process change that creates them and, where it helps, use unsupervised methods to get value in the meantime.

Frequently asked questions