Machine Learning for Insurance Pricing and Claims

Claims is usually the better place to start

Most conversations about machine learning in insurance begin with pricing, because pricing is where the money appears to be. We usually suggest starting with claims instead.

A managing general agent handling 2,000 property claims a year has a triage problem that costs money every day: simple claims wait in the same queue as complex ones, suspicious claims are spotted late, and experienced handlers spend time on paperwork a junior could clear. A model that routes claims on arrival is easier to validate, easier to explain and much less exposed to regulatory scrutiny than one that sets the price a customer pays.

Machine learning for claims triage and fraud

• Complexity scoring at first notification. Predicting which claims will settle quickly and which will need an adjuster, so fast-track claims are paid in days.
• Fraud referral. Ranking claims for the counter-fraud team based on timing, claim history, network links between parties and inconsistencies in the narrative.
• Document extraction. Pulling amounts, dates and descriptions from repair estimates, invoices and medical reports.
• Reserve estimation. Suggesting an initial reserve from similar past claims, which handlers can accept or adjust.
• Subrogation opportunities. Flagging claims where a third party is likely liable and recovery is worth pursuing.

Notice that none of these makes a final decision on a customer. A fraud score sends a claim to a person; it does not decline it. That design choice is what keeps the project defensible.

Insurance pricing models: GLMs, gradient boosting and the middle ground

Actuarial pricing has used generalised linear models for decades, and for good reasons: they are transparent, they behave predictably and regulators understand them. Gradient boosted models usually find more signal, especially interactions between rating factors, but they are harder to explain and easier to overfit on a small book.

The pragmatic pattern is to keep the GLM as the base rate and use machine learning to find the interactions and segments it misses, then feed the useful ones back as explicit factors. You get most of the lift while the final tariff stays something an actuary can sign off.

What regulation stops you doing

Price optimisation, meaning charging people what the model predicts they will tolerate rather than what their risk warrants, is where insurance modelling has caused the most harm. In the UK, the FCA's pricing rules for home and motor insurance now prohibit charging renewing customers more than equivalent new customers. Any model estimating willingness to pay should be treated with great suspicion.

The EU AI Act classes risk assessment and pricing for life and health insurance of individuals as high-risk, with obligations around data governance, documentation and human oversight. Fairness applies across all lines: a model that uses postcode or occupation as a proxy for a protected characteristic can produce discriminatory outcomes without anyone intending it.

Price on risk. The moment a model is pricing on how unlikely someone is to shop around, you have a conduct problem rather than a data science one.

When the book is too small to model

Credibility is the actuarial word for it. A small MGA with a few thousand policies and a few hundred claims a year does not have enough data to price with a complex model, and trying to do so will produce confident nonsense. In that situation, the useful work is data quality, claims triage and using reinsurer or market data for the base rates.

1. Clean up claims coding so that cause of loss is recorded consistently
2. Link policies, claims and payments with reliable identifiers
3. Build triage and document extraction, which need far less data than pricing
4. Revisit pricing models once two or three years of clean data have accumulated

Brokers have a different set of problems

Brokers do not price risk, but they do have plenty of prediction problems: which clients are likely to lapse at renewal, which quotes are likely to convert, which insurer a risk should be placed with and which renewals need a call rather than an email. These are classic propensity models, and they sit alongside the workflow improvements covered in our post on automation for insurance brokers.

The data usually lives in a broking platform with a patchy export, so the first weeks of any project go on getting reliable history out of it. That is normal and worth budgeting for.

Measuring whether the model is actually helping

Insurance outcomes arrive slowly. A claim triaged today may not settle for months, and a pricing change takes a full renewal cycle to show its effect on loss ratio. That makes it tempting to judge a model on the metrics that come back quickly, which are not always the ones that matter.

• For triage, track time to settlement on fast-tracked claims and how often handlers pull a claim back out of the fast track
• For fraud referral, track the proportion of referrals the counter-fraud team considers worth investigating, not just the volume
• For reserving, compare suggested reserves with final settled cost by claim type
• For pricing, watch loss ratio by segment and conversion together, since a price that wins every quote is probably too cheap

Agree these measures with the claims and underwriting leads before the model goes live. Afterwards, everyone has a favourite number.

How we would scope an insurance project

At SpiderHunts we ask for a year of claims data first, not a pitch deck. From that we can tell within a week or two whether triage is viable, what the fraud referral rate looks like and how consistent the coding is. We then build the model behind a workflow handlers already use, run it in shadow mode and compare its routing to what experienced handlers decided.

If you want the wider picture of what AI can do across an insurer, our overview of AI for the insurance industry covers the language model side, and our machine learning development page explains how we take a model from notebook to production.

Frequently asked questions