An AI Chatbot That Actually Answers Questions
Last updated:
Why people hate chatbots
Because most of them are worse than the alternative. They answer a question adjacent to the one asked, cannot escalate, and stand between a customer and a person who could have solved it in thirty seconds.
The technology is no longer the problem. The design is: a bot optimised to deflect tickets rather than to answer questions will always end up frustrating the people it deflects.
Grounding: answering only from what you have
The single most important design decision is that the bot answers from your content, not from the model's general knowledge. Your returns policy, your delivery times, your product specifications — retrieved, then answered from.
Concretely: the question is used to retrieve relevant passages from your own material, and the model is instructed to answer only from those passages and to say so when they do not cover the question. That constraint removes the majority of embarrassing answers.
An ungrounded bot asked about your refund window will produce a number. It will be a plausible number. It will not be your number.
Saying “I do not know” is a feature
A bot that answers 70% of questions well and admits the other 30% is far more valuable than one that answers everything with 85% accuracy. Users forgive not knowing; they do not forgive being told something wrong.
- Retrieved passages do not cover it — escalate
- Retrieval confidence is low — escalate
- The question involves an account, an order or money — check the real system or escalate
- The user asks twice — escalate without being asked
Escalation in one step
Handover must be one action, always visible, and it must carry the conversation with it. Making a frustrated customer repeat everything to a human is the moment the whole investment turns negative.
We also route on sentiment: a message expressing frustration goes to a person immediately regardless of whether the bot could have answered. The ticket deflection lost is worth less than the relationship saved.
What to measure
| Metric | What it tells you | Warning sign |
|---|---|---|
| Resolution rate | Answered without a human | High with low satisfaction = confident wrongness |
| Escalation rate | Handed to a person | Very low is suspicious, not good |
| “I do not know” rate | Honest gaps | Rising = content is out of date |
| Repeat questions | Same user asking again | The first answer missed |
| Post-chat satisfaction | Whether it helped | The only metric that matters alone |
Deflection rate on its own is a dangerous target. It rewards a bot for preventing contact rather than for resolving problems, and those are not the same thing.
What it takes to build and keep
A useful bot needs your content in a maintainable form, a retrieval layer, the answering logic with its refusal rules, escalation into your existing helpdesk, and a review process for the conversations it got wrong.
That last item is the one businesses under-plan. A chatbot is not a project that finishes; it is a system that needs someone reading a sample of conversations weekly and fixing the content gaps it exposes. Budget an hour a week forever, or expect quality to decay within two months.
Frequently asked questions
How long does a support chatbot take to build?
Can it access customer accounts and orders?
What happens when our policies change?
Will it handle other languages?
Want a chatbot that answers from your own content?
Tell us what your customers ask most and we will tell you how much of it can be handled well — and what should still reach a person.