What Features Are in Machine Learning, and Why They Decide Results

The columns decide the ceiling

Give two data scientists the same algorithm and the same customer records, and the one who builds better inputs will win by a distance. That is not an exaggeration for effect. On business data, the choice of features is usually the biggest single lever on how good a model becomes.

It is also the part of machine learning where business owners and managers can contribute most, because the best features encode things you already know about why customers buy, leave, pay late or complain.

What a feature is

A feature is one measurable fact about the thing you are predicting for. If the model predicts whether a gym member will cancel, each member is an example, and the features might be:

• Visits in the last 30 days
• Change in visits compared with the previous 30 days
• Months since joining
• Whether they have ever frozen their membership
• Price plan
• Distance from home postcode to the gym
• Classes booked but not attended

In a spreadsheet, features are simply the columns, apart from the answer column. The model looks at many members' features alongside whether they cancelled, and learns how the features relate to the outcome.

Raw data is not the same as good features

Your systems store raw events: a door swipe at 06:42 on a Tuesday, a payment on the first of the month. Models do much better with those events summarised into features that capture behaviour. Turning the first into the second is feature engineering.

Notice how many of the useful features are comparisons: this customer against their own history, this invoice against the usual. Those relative measures are often where the signal lives, and a model given only raw totals frequently misses them.

Where good feature ideas come from

Ask the people who do the work. A credit controller will tell you that customers who start paying in part are in trouble. A warehouse manager knows that orders placed after 3pm on Fridays get picked by agency staff and go wrong more often. A hotel's front desk knows which booking channels produce no-shows.

Each of those observations is a candidate feature. Some will prove predictive, some will not, and the testing is quick. Our first workshop on a SpiderHunts project is largely a structured version of this conversation, and it regularly produces the strongest inputs in the final model.

The algorithm finds the pattern. Somebody who has done the job for ten years usually knows where to look for it.

Traps that ruin otherwise good models

1. Features from the future. Using information only known after the moment of prediction, such as 'number of refunds this month' when predicting at the start of the month. This is leakage, and it produces wonderful test scores and useless live predictions.
2. Features that proxy for protected characteristics. Postcode or first name can stand in for ethnicity. If a model is used for decisions about people, check what your features are really measuring.
3. Features you cannot get live. A field populated monthly by a finance export is no use to a model that must predict in real time.
4. Identifiers. Customer numbers and order IDs let a model memorise individuals rather than learn patterns.
5. Too many weak features. Throwing in 400 columns in hope rather than reason invites overfitting and slows everything down.

The first trap deserves emphasis because it is so easy to fall into. For every feature, ask: at the exact moment this prediction is made, would we know this value.

Does deep learning make features obsolete

Partly, for some kinds of data. Neural networks working on images or text learn their own internal features from raw pixels or words, which is a big reason they took over those fields. You do not hand-craft 'has a scratch in the top corner' for a defect detection model.

For tabular business data, feature engineering still matters a great deal. And even in text-heavy projects, a common modern pattern is to use a language model to extract structured features, such as complaint type or urgency, which then feed a simpler model alongside your numeric data. The feature thinking has moved, not disappeared.

How features get tested and kept

Once a model is trained, standard tools show how much each feature contributed. That is useful for pruning weak inputs and, frankly, for sanity: if the most important feature is something nobody can explain, investigate before trusting the model.

• Keep feature definitions in code, shared by training and live prediction, so the two never drift apart
• Document what each feature means and where it comes from
• Monitor feature values in production, since a broken upstream field shows up there first
• Retire features that stop being available rather than filling them with defaults

If you are building on data spread across several systems, the pipeline that computes features reliably is often the biggest piece of work. That is where our data science and engineering sides meet. For a practical example of relative features at work, see forecasting demand with your own data, and for why the underlying records matter, our note on data quality prerequisites.

Frequently asked questions