The invoice from the data provider
Your product enriches properties with data: sold prices, EPC ratings, flood risk, planning history, school catchments, rental estimates. Some comes from paid providers charged per lookup. When you launched, the bill was small. Now it rises faster than revenue. Nobody is sure why.
A developer digs in and finds the property page calls three APIs every time it loads, even for a property viewed a hundred times this month. The valuation tool repeats the same comparables lookup for every tweak a negotiator makes. One agency runs a bulk report every night that calls the API for every property on their books. None of this was a decision anyone made.
How data costs creep up
Data costs start small, so they are built without much care. Then usage grows unevenly and the bill grows faster.
- Lookups happen on screen load rather than when data is actually needed.
- Results are not stored, so the same property is looked up again and again.
- Open datasets that cover the same ground, such as HM Land Registry Price Paid Data and EPC data, are not used first.
- Heavy users and bulk features are not visible in the cost, so pricing does not reflect them.
- Nobody owns the data bill, so nobody watches it.
What uncontrolled data spend does
Your gross margin falls as you grow, which is the opposite of what investors and your own plan expect. You may be losing money on your heaviest agencies without knowing it. Developers start avoiding features that need data, because nobody knows what they will cost. And if a provider changes its prices, you have no way to measure the effect until the invoice arrives.
It can also shape your pricing in the wrong direction. Without cost per agency, you tend to price plans on branch count alone, while the real driver of cost is how a particular agency uses the data features. A small agency running bulk reports every night can cost more to serve than a large one that barely opens the valuation tab, and both pay what their plan says.
How we build a data layer with a budget
What we build puts all external property data behind one layer that knows what it costs.
- A single data service your product calls instead of calling providers directly, so every lookup is counted and controlled in one place.
- Caching with a sensible lifetime per data type: sold prices and EPCs change rarely, rental estimates more often, and each is refreshed on its own schedule.
- Open data first: price paid, EPC and other public datasets loaded into your own database where their licences allow, with paid providers used only for what they add.
- Lookups made when data is needed, such as when a negotiator opens the valuation tab, not every time a list of properties renders.
- Cost tracking per feature and per agency, so you can see which features and customers drive the bill.
- Budgets and alerts: a monthly limit per agency or plan, a warning as it approaches, and throttling for bulk jobs that would exceed it.
| Data type | Changes how often | How we source it |
|---|---|---|
| Sold prices | Rarely for a given sale | Open data in your own database |
| EPC ratings | When a new certificate is lodged | Open data, refreshed on a schedule |
| Rental estimates | Often | Paid provider, cached for a short period |
| Flood or planning data | Occasionally | Paid or open, cached per property |
| Comparables for a valuation | Per request | Built from your stored data where possible |
Checking data licences for what you may store and show is your responsibility with your provider. We design the layer to respect whatever terms you have.
The bill after the change
The property page now reads from your own store and only calls a provider when stored data is out of date. The valuation tool fetches comparables once per session. The agency's nightly report runs against stored data, and only calls providers for properties that have changed. Your finance lead can see data cost per agency next to revenue per agency, and the pricing discussion about bulk reports becomes a conversation with numbers rather than a guess.
Are data costs outrunning revenue?
- Your data provider bill grows faster than your revenue.
- The same property is looked up many times a month.
- You do not use open datasets for sold prices or EPCs.
- You cannot say which feature or agency drives data cost.
- Nobody would notice a sudden rise in lookups until the invoice.