Great on stage, awkward in the wild
The feature looked impressive when you showed it: summarise a document, draft a reply, answer a question about the customer's data. You shipped it. Now the feedback is mixed. Some answers are excellent. Some are confidently wrong. One customer got a summary that mentioned something that was not in the document. Response times vary. The model provider's bill is higher than anyone expected.
The team tweaks the prompt when someone complains, and each tweak fixes one case and seems to break another. Nobody can say whether the feature is getting better or worse overall.
Why AI features behave differently in production
A demo uses examples chosen because they work. Production uses whatever customers throw at it: long files, scanned PDFs, half-filled records, unusual questions, other languages. The model has not changed. The inputs have.
- No test set, so there is no way to measure quality before and after a change.
- The model answers from general knowledge instead of the customer's own data.
- Inputs are passed in raw, including irrelevant or badly extracted text.
- No checks on the output before it is shown or acted on.
- No limits on length or retries, so costs climb unpredictably.
- No fallback when the provider is slow or unavailable.
What an unreliable AI feature costs
| Problem | Effect on the product |
|---|---|
| Confidently wrong answers | Customers stop trusting the feature, then the product |
| Prompt changes without testing | Fixes that silently break other cases |
| Unpredictable costs | Margins on AI-heavy customers shrink or vanish |
| Slow responses | Users abandon the feature before it finishes |
| Data concerns | Enterprise customers ask where their data goes and you are unsure |
How we make an AI feature reliable
- Build an evaluation set. We collect real, anonymised examples from usage, including the failures, with what a good answer looks like, and score the current feature against them.
- Ground it in the customer's data. Retrieval from their own records and documents, scoped strictly to that customer, so answers come from facts the product holds.
- Clean the inputs. Better text extraction, trimming irrelevant content, and structured context instead of raw dumps.
- Check the outputs. Structured output formats, validation rules, citations back to source records where it helps, and refusing to answer when the data does not support one.
- Keep a person in the loop where it matters. Drafts are presented as drafts, and actions with consequences need confirmation.
- Control cost and speed. Choosing the right model per task, caching repeated work, limiting context size and setting usage limits per customer.
- Plan for outages. Timeouts, retries and a fallback provider or graceful message when the model is unavailable.
- Monitor in production. Quality sampling, user feedback buttons, latency and cost per customer, tracked over time.
We work with models from OpenAI, Anthropic Claude and others through a thin abstraction, so switching models is a tested change rather than a rewrite. Every prompt or model change is run against the evaluation set before it ships.
An AI feature you can vouch for
The team's way of working changes most. Instead of reacting to the latest complaint with a prompt edit, someone proposes a change, runs it against the evaluation set, and ships it only if the scores hold up across the whole set. Failures from production are added to the set, so the same mistake is tested for from then on.
You can say whether the feature got better after a change, with evidence. Wrong answers become rarer and, when the data is not there, the feature says so. Costs per customer are known and bounded. And when an enterprise customer asks how their data is used by the AI, there is a clear, accurate answer.
Is your AI feature in this state?
- The feature worked well in demos and is patchy with real customers.
- Prompt changes are made in response to complaints, without testing.
- You have no set of examples to measure quality against.
- AI costs per customer are unpredictable.
- The feature can answer from general knowledge rather than the customer's data.