Most A/B tests end after the moment has passed
A retailer runs a four-week A/B test on two versions of a seasonal banner. By the time the result is significant, the season is nearly over and half the traffic spent a month looking at the loser. For long-lived page elements that is a fair price for a clean answer. For a promotion that lasts ten days, it is mostly waste.
This is the gap dynamic content driven by machine learning fills. Rather than splitting traffic evenly until a winner is declared, an algorithm adjusts in real time, sending more visitors to what is working while still testing the alternatives a little. We covered the rules-based side of this in AI website personalisation; this piece is about the learning algorithms underneath.
How a multi-armed bandit works
The name comes from a gambler facing a row of slot machines with unknown payouts. Each pull teaches you something about one machine. The question is how to balance exploring machines you know little about with exploiting the one that looks best so far.
On a website, each variant is an arm and each visit is a pull. Common approaches include:
- Epsilon-greedy. Show the current best variant most of the time and a random one, say, 10 percent of the time. Simple and easy to explain.
- Thompson sampling. Keep a probability estimate for each variant and pick in proportion to the chance it is best. Efficient and the usual default.
- Contextual bandits. Take visitor context into account, such as device, traffic source, new or returning, so different visitors can get different winners.
Bandit or A/B test: how to choose
| Situation | Better choice | Why |
|---|---|---|
| Seasonal or short promotion | Bandit | Earns during the test instead of after it |
| Headline for a news or offer page that changes often | Bandit | No time for a significance test per item |
| Checkout redesign | A/B test | You need a reliable estimate of the effect and to understand side effects |
| Pricing change | A/B test, carefully | Long-term effects and fairness matter more than quick gains |
| Many variants, e.g. ten hero images | Bandit | Drops poor performers quickly |
| Low traffic page | Neither, usually | Use judgement and qualitative research |
The honest trade-off is that bandits optimise, while A/B tests measure. A bandit will not tell you with confidence how much better the winner is, and it can settle early on a variant that got lucky in its first days. If the decision is permanent and expensive, use a proper test. Our note on A/B testing for small traffic sites covers the measurement side.
Contextual bandits and what they need
Contextual bandits are where dynamic content starts to look like personalisation. A returning customer from an email campaign might see a loyalty offer, while a first-time visitor from a paid search ad sees a trust message and delivery information.
They need considerably more traffic, because the algorithm is effectively learning a separate answer for each context. A useful rule of thumb: keep contexts few and meaningful. Device type, new versus returning, and traffic channel are often enough. Twenty contextual features on a site with 30,000 monthly visitors will learn noise.
An illustrative setup
Imagine a holiday lettings company with 80,000 monthly visitors that changes its homepage offer every two weeks. Each fortnight, marketing produces four banner variants. A Thompson sampling bandit allocates traffic, with the reward defined as a started booking enquiry within the session rather than a click, because clicks reward curiosity rather than intent.
- Define the reward carefully and log it against the variant shown
- Keep a small fixed holdout, perhaps 5 percent, showing a default banner, so overall lift can be measured
- Set a minimum exploration share so no variant is starved in the first day
- Reset or decay learning when the offer changes, because last fortnight's winner is irrelevant
- Review weekly which variants won and share the patterns with the people writing the copy
That last step matters. The algorithm finds winners; people learn why, which makes the next batch of variants better.
Where dynamic content goes wrong
- Optimising the wrong reward. Click-bait variants win on clicks and lose on sales.
- Novelty effects. A new design does well briefly because it is new; bandits can lock in on it.
- Caching. Pages served from a CDN cache can show everyone the same variant, silently breaking the allocation.
- Flicker. Swapping content with client-side JavaScript after the page loads looks broken and can hurt Core Web Vitals. Decide server-side or at the edge where possible.
- SEO confusion. Keep indexable text and headings stable; vary banners and offers rather than the core content search engines read.
Build, buy or skip
Many experimentation platforms now include bandit allocation, and for most businesses buying is sensible. Building makes sense when the content decision is core to your product, such as ranking listings or choosing offers inside an app, and when you want control over the reward and the data.
Skip it if your traffic is low or your content rarely changes. When SpiderHunts builds these systems as part of machine learning projects, the algorithm is a few hundred lines; the logging, the reward definition and the holdout are what decide whether anyone can trust it.