The pile nobody reads
Most businesses collect far more free-text feedback than anyone reads: review sites, post-purchase surveys, NPS comments, support conversations, cancellation reasons. It is the most direct information available about why customers behave as they do, and it usually sits unused.
The instinct is to run topic modelling across the lot and see what emerges. That produces something, but rarely something anyone can act on.
Why unsupervised themes disappoint
Unsupervised methods group text by statistical similarity, with no knowledge of what matters to your business. The clusters that come back tend to split on vocabulary rather than meaning, and mix issues you would never put together.
A typical output is a cluster containing delivery complaints, packaging complaints and a few comments about the courier's manner - related by words, but owned by three different teams with three different fixes.
They are also unstable. Re-run next quarter with new data and the clusters shift, so you cannot track whether a theme is growing.
Define the themes, then classify
The approach that works is less exciting and considerably more useful: decide what you need to know, then classify each comment against those categories.
- Read a couple of hundred comments by hand. This is not optional and it is always worth the afternoon.
- Write a theme list tied to who would act on each one - delivery speed, packaging condition, product fit, pricing, website usability.
- Label a sample against that list, checking that two people agree on the labels.
- Train a classifier, or use a language model with the definitions supplied, and measure it against the held-out labels.
- Track theme volumes over time, which is where the value actually is.
Sentiment alone tells you nothing
A sentiment score is not a finding. Knowing 22% of comments are negative does not tell anyone what to change, and the number moves for reasons that have nothing to do with your product.
Theme plus sentiment is actionable: negative comments about delivery timing rose this month while negative comments about product quality did not. That points at a specific team and a specific question.
| Output | Can someone act on it? |
|---|---|
| Overall sentiment score | No |
| Sentiment by theme | Yes |
| Theme volume over time | Yes |
| Theme by product or region | Yes, and usually the most useful cut |
Keep the unsupervised pass for discovery
Clustering still has a role: finding themes your list does not contain. Run it periodically over comments the classifier found hard to place, and read what comes out.
That is the right division of labour - structured classification for tracking, clustering for discovery. Used that way the awkward cluster is a feature, because it is where a new problem shows up first.
Comments nobody reads are not feedback. They are storage.