Image Classification for Product Catalogues

Where catalogue tagging actually hurts

A fashion wholesaler adds 1,500 lines a season from 40 suppliers. Each supplier sends a spreadsheet with its own idea of what a category is, half the colour fields are blank, and one sends everything as 'Misc'. Someone in the office spends three weeks fixing it before the site can go live.

That is the job image classification is good at. The photo usually contains more reliable information than the supplier's spreadsheet. A model can look at it and say this is a midi dress, long sleeved, navy, floral print, and be right most of the time.

It will not be right all of the time, and designing for that is most of the work.

What a model can tag from a product photo

Anything that can only be known from the label or the spec sheet — fabric composition, dimensions, compatibility — should come from the text, not the image. Combining both in one pipeline usually beats either alone.

Fine-tune a model or use a general one?

Two routes exist, and the right one depends on how unusual your catalogue is.

General multimodal models can describe a product photo against a list of categories you give them, with no training. That is fast to trial and fine for common retail categories. The downsides are per-image cost at volume, less consistency between runs, and weaker results on specialist products such as industrial fittings or spare parts that all look like grey metal.

A fine-tuned image classifier, built on a pretrained vision model and trained on a few thousand of your own labelled photos, is cheaper to run, faster, and more consistent. It takes more set-up and needs retraining when the taxonomy changes. For a stable catalogue above tens of thousands of images a year, it is usually the better long-term choice. Our machine learning service typically starts with the general route to prove the value, then moves the high-volume categories to a trained model.

Fix the taxonomy before you train anything

The quiet killer of these projects is a category tree nobody agrees on. If 'Outerwear' and 'Coats & Jackets' both exist, and staff have used them interchangeably for five years, the model will learn that confusion faithfully.

1. Export the current category tree and product counts per node
2. Merge overlapping categories and delete ones with a handful of products
3. Write a one-line definition per category, with a borderline example
4. Relabel a sample of 500 products against the new tree by hand
5. Only then train or configure the model

Step four doubles as your test set. If two people disagree on 15% of the sample, the model cannot be expected to beat that.

The review queue is the product

Every prediction should carry a confidence score. Above a threshold it goes straight onto the product. Below it, a person sees the image, the suggested tags and one-click alternatives.

Illustratively, if 80% of a new supplier feed clears the threshold and a reviewer handles the rest at ten seconds each, a 1,500-line intake drops from weeks to an afternoon. The reviewer's corrections feed back into training, so the threshold can come down over a few seasons.

Automating 80% well beats automating 100% badly. The last 20% is where the returns and the angry emails come from.

When image classification is the wrong tool

• Small catalogues. Under a few hundred products with slow turnover, a person tagging carefully is cheaper and better.
• Products that look identical. Screws of different thread pitch, cables of different rating. The difference is in the spec, not the pixels.
• Poor photography. Supplier images on cluttered backgrounds with inconsistent lighting will cap accuracy. Improving product photography may be the better first spend.
• Regulated attributes. Allergens, safety ratings and age grades must come from verified data, never an image guess.

Measuring whether it is working

Track accuracy per category rather than one headline number. A model that is 95% right overall can be 60% right on the category that matters most. Also watch the share of items routed to review each week, reviewer override rates, and on-site signals such as filter usage and zero-result searches, which improve when attributes are complete.

When we build these at SpiderHunts we show a confusion table per category on a simple internal page. It makes the model's weak spots obvious to the merchandising team without anyone needing to read a metric.

Expect drift. Suppliers change photographers, a new season brings a trend the model has never seen, and a category that scored well in spring slips by autumn. A monthly check of a fresh hand-labelled sample catches that before the filters on your site start quietly lying to shoppers. It takes a merchandiser an hour, and it is the cheapest insurance in the whole system.

If you are weighing this against other catalogue and merchandising projects, our overview of AI for retail and ecommerce puts it alongside recommendations, pricing and demand forecasting so you can see which is likely to pay back first.

Frequently asked questions