Uplift Modelling: Targeting Customers Who Will Actually Change Their Mind

Four kinds of customer, and only one worth paying for

Every campaign reaches a mix of people. Uplift modelling starts by admitting it.

Traditional targeting, including most propensity-to-buy models, happily sends the offer to sure things because they score highest. The campaign report looks excellent. The margin quietly suffers.

Sleeping dogs are real

The fourth group sounds theoretical until you see it. A subscription business emails 'your contract is up for renewal, here is a loyalty discount' to customers who had forgotten they were paying. Some of them take the reminder as a cue to shop around. A mobile network's retention call prompts a dormant customer to check competitors. A 'we miss you' email lands on someone who unsubscribed in spirit months ago and now does so formally.

An uplift model is the only standard technique that can identify these customers, because it is the only one that compares treated and untreated behaviour directly.

What uplift modelling needs

The requirement is non-negotiable: randomised experiments. To learn the effect of an action, you need customers who were randomly chosen to receive it and similar customers randomly chosen not to.

• A holdout group in past campaigns, chosen at random rather than 'people we forgot to include'
• Enough customers in both groups, usually thousands, because uplift effects are smaller and noisier than purchase rates
• Outcomes tracked for both groups over the same period
• Customer attributes as they were before the campaign
• Consistency: the same kind of offer, since uplift from a 10% discount says little about free delivery

If you have never run holdouts, the first project is not a model. It is changing how campaigns are sent so every one includes a random control group. That costs a little revenue in the short run and pays back with the first honest measurement.

How the model works, briefly

There are several technical approaches. The simplest trains two models, one on treated customers and one on the control group, and takes the difference in predicted outcome as the uplift. More refined methods model the treatment effect directly, which tends to be more stable with limited data. Open-source Python libraries implement the common ones, so there is no need for exotic tooling.

Evaluation is different from ordinary models too. You cannot see an individual's uplift, because nobody both received and did not receive the offer. Instead you rank customers by predicted uplift and check, on held-out experimental data, whether the treated-minus-control difference really is larger at the top of the list. The standard chart for this is an uplift or gain curve.

A worked illustration

Imagine an online homeware retailer sending a 15% discount to 100,000 lapsed customers, with a random 10% held out. Among the treated, 4% buy; among the control, 3% buy anyway. The campaign's true incremental effect is one percentage point, and three-quarters of the discounted orders would have happened without the discount.

An uplift model trained on that experiment might show that customers with a recent product view and no purchase in four months respond strongly, while those who bought in the last sale period barely change. Sending the discount only to the top 30% by predicted uplift could keep most of the incremental orders at a fraction of the discount cost. These numbers are for illustration, but the fact that a large share of discounted orders are sure things is the rule rather than the exception.

When uplift modelling is not worth it

1. Small customer bases, where experiments cannot produce enough treated and control customers to detect differences
2. One-off campaigns that will never be repeated, since the model learns about a specific action
3. Actions with no real cost, like a routine newsletter, where precision targeting saves little
4. Organisations unwilling to hold out customers from campaigns at all

In those cases, simple holdout testing of whole campaigns still tells you whether the campaign as a whole was worth running, which is more than most businesses know.

Where to start

Pick the most expensive recurring action you take on customers: a retention discount, a win-back voucher, a sales call. Start randomising it with a control group now. After two or three rounds, there is usually enough data for a first uplift model. At SpiderHunts we often spend the first month of an uplift project on campaign plumbing rather than modelling, and our machine learning team considers that the most valuable part. For the retention side specifically, pair this with churn prediction so you know both who is at risk and who can be saved.

Frequently asked questions