Building a Machine Learning Model With Only a Few Hundred Examples

Small data is normal, not a failure

Most of the machine learning advice online is written by people with millions of records. A regional insurer with 400 past claims disputes, a manufacturer with 300 logged machine faults or a recruiter with 600 placements does not live in that world, and does not need to.

Plenty of valuable business models are built on a few hundred examples. What changes with small data is the method. You need approaches that learn from little, and you need to be much more careful about believing your own results.

Approaches that work with small datasets

• Simple, well-regularised models. Logistic regression and shallow decision trees learn stable patterns from few examples and are easy to explain.
• Gradient boosting with restraint. Tools such as XGBoost or LightGBM work on small tabular data if you limit tree depth and number of features.
• Pretrained models plus a small classifier. For text or images, turn each item into an embedding with a general model and train a simple classifier on top. The heavy lifting was done elsewhere.
• General AI models with examples in the prompt. For classification and extraction, a language model shown a handful of labelled examples may perform well with no training at all.
• Domain rules as features. A rule an expert already uses, encoded as an input, is worth hundreds of rows.

What does not work is training a deep neural network from scratch or throwing 200 columns at 300 rows. With that many inputs, the model will find patterns that exist only by chance.

Rough guide to what each approach needs

These figures are illustrative starting points from our own experience, not thresholds. The complexity of the pattern matters as much as the count.

Testing honestly when data is scarce

With 300 examples, a 20% test set is 60 items. Get three more right by luck and accuracy jumps five points. That is why small-data projects so often look great in development and disappoint later.

1. Put aside a final test set at the start and do not look at it until the very end
2. Use cross-validation on the rest, training and testing on several different splits and looking at the spread of results, not just the average
3. Compare against a simple baseline, such as always predicting the most common answer or the rule staff already use
4. Report a range, not a single number: 'somewhere between 72% and 84%' is honest and useful
5. If your data spans time, test on the most recent period rather than a random sample

If the model cannot clearly beat the baseline across the cross-validation splits, it has not learned anything reliable, however good one run looked.

Making every example count

When examples are scarce, effort is better spent on quality and choice than on algorithms. Check every label. A dozen mislabelled rows out of 300 is a serious amount of noise.

Spend time with the people who make the decision today and ask what they look at. The features they describe are usually the ones that carry the signal. A claims handler who says 'disputes almost always involve a third-party garage' has just given you your best input.

If you can label more, label cleverly. Active learning means training a rough model, then asking a person to label the examples the model is least sure about. Those teach far more than random ones. Some teams also generate synthetic examples, which can help for rare categories but can easily teach the model an unrealistic version of the world; our post on synthetic data for machine learning covers the trade-offs.

When a model is the wrong answer

Sometimes a few hundred examples reveal that the pattern is simple enough to write down. If a shallow tree turns out to be three rules, implement the three rules in your software and skip the model entirely. That is cheaper to run and easier to trust.

Other times the data shows there is no stable pattern yet, and the honest move is to keep collecting for another six months. For text-heavy tasks, the answer is increasingly to use an existing AI model rather than train your own, which is the kind of work covered by our AI integration service.

How we approach small-data projects

At SpiderHunts, a small-data project starts with a baseline, a locked test set and a conversation with the people who make the decision. We try the simplest credible approaches first and only add complexity when it clearly beats them across repeated tests.

We also plan for the dataset to grow. A model built on 300 examples should be retrained as the next 300 arrive, and the pipeline should make that routine. If you want to explore whether your problem is a fit, our machine learning team is happy to look at the data first. It is also worth running through our data readiness checklist beforehand.

Frequently asked questions