Training Data, Explained for Non-Technical Founders

Your product's taste is set by what it was fed

Founders building a machine learning feature tend to worry about the model. Which algorithm, which provider, whether to hire a specialist. Those choices matter far less than the one they tend to leave to someone else: what data it learns from.

The training data sets the limits of everything. A model cannot be more accurate than the data allows, cannot handle situations the data never showed it, and will faithfully reproduce any mistakes baked into it. If there is one part of the technical side a non-technical founder should understand properly, it is this one.

What training data looks like in practice

For most business products it looks like a big spreadsheet. Each row is one example: one customer, one transaction, one delivery, one support ticket. The columns describe that example, and one special column holds the answer you want the model to learn to predict.

Say you are building a tool that predicts whether a restaurant booking will be a no-show. Each row is a past booking. Columns include party size, day of week, how far in advance it was made, whether a deposit was taken, and whether the guest has booked before. The answer column says whether they turned up.

For language and image products the examples are documents, photos or recordings with labels attached, but the principle is identical: examples plus answers.

Five ways to judge the data you have

The fourth row catches out even experienced teams. If 'number of complaints' is filled in after a booking ends, a model using it will look superb in testing and be useless live, because at booking time the number is unknown. This is called leakage, and it is the single most common reason a promising prototype collapses.

Where startups actually get their data

A startup often has little history of its own. The realistic sources, roughly in order of how often we see them work:

1. Pilot customers' historical records, used under a clear agreement, often in exchange for a discount
2. Your own product's usage, collected deliberately from day one with the outcome fields designed in
3. Manual labelling by domain experts on a few thousand examples
4. Public or licensed datasets for general patterns, fine-tuned later on your own data
5. Synthetic data to fill gaps, used carefully and never as the only source

On the last point, our article on synthetic data explains where it helps and where it quietly misleads.

A good strategy for many startups is to launch the product with simple rules, instrument it heavily, and switch on learning once real usage has produced enough examples. Investors tend to respect that more than a model trained on borrowed data that does not match your market.

The legal side founders skip

If the data describes people, UK and EU data protection law applies, and using personal data to train a model is a purpose you need a lawful basis for. Data collected to fulfil an order is not automatically available to train a product sold to other companies.

• Check your privacy notice actually covers model training
• Put training rights explicitly into customer contracts, especially for B2B data
• Remove or pseudonymise identifiers you do not need
• Record where each dataset came from and under what terms
• Check whether the EU AI Act classes your use case as high risk, as data governance duties then apply

None of this is legal advice, and a solicitor familiar with data protection is worth an hour of your time. It is far cheaper than retraining a model because a client withdrew consent for their data.

Bias lives in the data, not the algorithm

If past hiring managers favoured certain CVs, a model trained on their decisions learns that preference. If a lender historically declined applicants from particular postcodes, the model does the same, only faster and more consistently.

A model trained on past decisions is a very efficient way of repeating them. Make sure they were decisions worth repeating.

The fix starts with looking at the data by group before training, and deciding deliberately what the model should learn from. That is a founder-level decision, not a technical detail.

How much to invest in data, and when

Early on, spend on collection design rather than volume: make sure the outcome is captured, cleanly, every time. That costs a few days of product work and pays for itself for years. Later, budget for a proper data pipeline, because the training data will need refreshing as the product grows.

When SpiderHunts helps a founder plan a SaaS product with a learning feature, we design the data capture before the model. It is unglamorous and it is the part that decides whether the model is ever any good. If your data quality is already a concern, this piece on data quality problems is a useful next read.

Frequently asked questions