Machine Learning for Energy and Utilities

Smaller players, same forecasting problem

Large utilities have had forecasting teams for decades. The newer entrants do not: independent energy suppliers, community energy schemes, solar and battery operators, heat network operators, water retailers and EV charging networks. They face the same core question as the incumbents, which is what demand and supply will be tomorrow and next month, with a fraction of the staff.

Consider a regional supplier with 40,000 customers buying wholesale power. Every half-hour it gets wrong costs imbalance charges. A few percent better forecast accuracy on that position is often worth more than the entire cost of the model, and the arithmetic is visible on the next settlement statement.

Machine learning use cases in energy and utilities

• Demand forecasting. Half-hourly or hourly load by customer segment, driven by weather, calendar effects and the growing share of heat pumps and EVs.
• Generation forecasting. Solar and wind output from weather forecasts, needed for trading and for battery dispatch.
• Battery and flexibility optimisation. Forecasts feeding an optimiser that decides when to charge, discharge or bid into flexibility markets.
• Asset failure prediction. Transformers, pumps, inverters and chargers flagged before failure using sensor and maintenance history.
• Leak, fault and theft detection. Anomalies in flow, pressure or smart meter readings that suggest a leak, a faulty meter or tampering.
• Arrears risk. Identifying customers heading into payment difficulty so support can be offered early.

Forecasting first, and how to judge it

Forecasting is where we tell most energy businesses to start, because it has a clear score. Measure the error of your current method over the last year, then measure the model on the same period without letting it see the answers. If the model does not win clearly on the periods that cost money, typically peaks and cold snaps, it does not ship.

Two lessons repeat. First, the weather forecast you pay for often matters more than the model. Second, forecasts should come with ranges, not single numbers, because a trader decides differently when the uncertainty is wide. Our guide to time series forecasting for demand planning covers the method in more depth.

Anomaly detection on meters and networks

Smart meter and network sensor data suits anomaly detection: a lot of readings, most of them normal, and the valuable ones rare. A water retailer might flag business premises whose overnight consumption never drops to zero, a strong hint of a leak. A charging network might spot chargers whose session failure rate creeps up days before they stop working.

The hard part is alert volume. A model that flags 2% of 50,000 meters produces 1,000 alerts, and nobody will investigate 1,000. We tune for the number your team can actually handle and rank by likely value. Anomaly detection for business operations explains that tuning trade-off.

Customers, vulnerability and regulation

Arrears prediction is useful and sensitive. Used to offer payment plans and support earlier, it helps customers and reduces bad debt. Used to decide who gets disconnected or pushed onto a prepayment meter, it is exactly the kind of system regulators scrutinise.

• Use the score to trigger help, never as the sole basis for an adverse decision
• Check outcomes by group, including customers flagged as vulnerable
• Keep human review for any action that restricts supply
• Document the model, its data and its limits, as the EU AI Act expects for higher-risk uses

Parts of utility infrastructure also fall under the Act's high-risk categories where AI acts as a safety component of critical infrastructure. Most forecasting and back-office models do not, but the classification should be checked rather than assumed.

Costs and when to hold back

Indicative ranges: a benchmarked demand or generation forecast, six to ten weeks; forecasts integrated with trading or battery dispatch, three to five months; asset failure models, heavily dependent on failure history.

Hold back if you have very few recorded failures, if your settlement data is not yet reconciled, or if a bought-in forecasting service already performs well for your portfolio. Buying is often right for generation forecasts. Building makes sense when your customer mix or assets make you unusual.

Asset failure models deserve extra caution. A heat network operator with 30 plate heat exchangers might have seen four failures in five years. That is not a dataset, it is an anecdote. In that situation, condition monitoring with sensible thresholds and a good maintenance log is the right answer today, and the log becomes the training data for a model in a few years. Saying so is less exciting than selling a predictive platform, and considerably cheaper for you.

How we approach it

SpiderHunts builds energy models the way a trader would want them: backtested against your historic positions, with uncertainty ranges, and monitored every day after launch because weather patterns and customer mix drift. If you are exploring this, our data science services describe how we run the benchmark stage before any production build.

Frequently asked questions