Collecting the Right Data Now for the Model You Will Want Next Year
Last updated:
You cannot go back and record last year
The most common reason we tell a business its machine learning idea will have to wait is not a lack of data. It is that the data they have does not record the thing that matters. The outcome was never logged, or the history was overwritten every time a record changed.
That is frustrating, because the fix is usually small. A dropdown added to a form, a status change written to a log table, a timestamp that was not being saved. The catch is that it only helps from the day you make the change. Every month you wait is a month of history you will not have.
So if a model is on your list for next year, the best thing you can do this month is not buy a platform. It is change what you record.
Record outcomes explicitly
Models learn from outcomes. Many systems record activity but not results. A quoting system stores every quote sent but not whether it was won, lost or simply ignored. A support desk closes tickets but does not record whether the fix worked or the customer came back a week later.
For each prediction you might want, ask what outcome would count as the answer and make sure it is stored as a clear field with a date. 'Lost' is useful. 'Lost, to a competitor, on price, on 12 March' is far more useful.
Keep history instead of overwriting it
Most business software stores the current state of a record. When a lead moves from 'contacted' to 'qualified', the old value disappears. When a customer changes segment, their past segment is gone.
For machine learning, the journey often matters more than the destination. How long did the deal sit at each stage? When did the customer's order frequency start dropping? You can only answer those questions if changes were recorded.
- Turn on field history or audit logging in your CRM for key fields, if it offers it
- In custom systems, write status changes to an event table with a timestamp and who made the change
- Where neither is possible, take a daily snapshot of the important tables
- Never delete records that represent real events; mark them inactive instead
An event log is also useful for everyday reporting and auditing, so this is rarely wasted even if the model never happens.
Capture reasons as choices, not paragraphs
Free text is where good data goes to hide. 'Why did this customer cancel?' answered in a notes box produces 'too expensive', 'price', 'moving to X', 'see email' and a lot of blanks.
Add a required dropdown with a sensible list of reasons plus 'other', and keep the free text box for detail. Review the 'other' answers every few months and add new options when patterns appear. It is a small change for staff and turns an unusable field into one of the most valuable inputs you will have.
A dropdown added today is worth more to next year's model than any algorithm chosen next year.
Log what you decided and offered
This is the one almost everybody misses. If you want to predict which discount wins a deal, you need to know which discounts were offered, including on the deals that were lost. If you want to learn which customers respond to a retention call, you need to record who was called and who was not.
Without a record of the decision, a model can only learn from outcomes, not from what influenced them. Worse, it may learn the effect of your existing policy and mistake it for customer behaviour. Recording decisions and, where practical, occasionally varying them, makes later analysis far more trustworthy.
What to start capturing for common future models
| Model you might want | Start capturing now |
|---|---|
| Customer churn prediction | Cancellation date and reason, support contacts, usage or order history kept over time |
| Quote win prediction | Outcome of every quote, reason lost, discount offered, competitor if known |
| Late payment prediction | Invoice issue, due and actual payment dates, reminders sent, disputes |
| Demand forecasting | Stock-outs, promotions with dates, price changes, lost sales where known |
| Predictive maintenance | Fault dates, fault type from a fixed list, parts replaced, run hours or sensor readings |
| Lead scoring | Lead source, every stage change with date, final outcome including no response |
Collect with care, not greed
There is a tension here. Data protection law requires you to collect only what you need for a stated purpose, and 'we might build a model one day' is not a strong justification for hoarding personal data.
The good news is that most of the changes above are about business events, not more personal information. Outcomes, reasons, dates and decisions rarely need more than you already hold. Where a future model would use personal data in a new way, record the purpose now, update your privacy notice if needed, and set retention periods.
- List the two or three predictions most likely to matter next year
- For each, write down the outcome and the inputs you would need
- Check which are already recorded with history
- Make the small system changes for the gaps
- Check in after three months that the new fields are actually being filled in
Step five matters. A new dropdown that staff skip is no better than no dropdown. At SpiderHunts, when we make these changes as part of our custom software work, we add a simple completeness report so gaps get noticed early. When you are ready to assess the data, our machine learning data readiness checklist is the natural next step, and data strategy for businesses covers the wider planning.
Frequently asked questions
What data should I collect for machine learning?
How long should we collect data before building a model?
Is it legal to collect data for future AI projects?
Why does overwriting data in a CRM cause problems for machine learning?
Should we use free text or dropdowns for reasons?
Thinking about machine learning next year?
Tell us which predictions you might want. We will suggest the data capture changes worth making now, most of which are small tweaks to systems you already use.