AI Voice Agents: An Honest Assessment

What changed and what did not

Voice quality and latency have improved to the point where a well-built agent no longer sounds obviously synthetic, and interruption handling has improved enough that conversations flow. That is genuinely new.

What has not changed: understanding a distressed customer, handling heavy background noise, or negotiating anything with nuance. Those remain human work and pretending otherwise damages your brand rather than your cost line.

A real deployment

We built an AI voice sales agent for a call centre using ElevenLabs for speech, Claude for the conversation and Salesforce for CRM. The outcome: headcount on that function went from 60 to 4 monitors, calling ran round the clock, volume tripled, and net savings exceeded $14,500 a month.

The important detail is the four monitors. They were not decorative. Someone has to listen to samples, handle escalations and correct the script when the agent starts saying something unhelpful — that role is the difference between a system that works and one that embarrasses you.

Where voice agents work today

• Outbound qualification at volume — structured questions, consistent delivery, no fatigue at call four hundred
• Appointment reminders and confirmations, with rebooking handled in the call
• Simple inbound triage — identify the caller, understand the need, route with context
• Status and account queries that need a lookup and a clear answer
• Out-of-hours coverage, where the alternative is voicemail nobody returns

Where they still fail

Emotional calls, where a caller is upset and needs to feel heard. Complex negotiation. Heavy accents or noise the model handles poorly. Anything requiring judgement about an exception to policy.

Design for these rather than hoping. Detect frustration and hand over immediately; detect repeated misunderstanding and hand over; make the route to a person obvious at all times.

Be transparent, and say it early

Disclose that the caller is speaking to an AI assistant, at the start. Beyond any regulatory considerations, it sets expectations correctly and people are markedly more tolerant of an imperfect AI than of a human they suspect is being faked.

Regulations around automated calling, consent and recording vary by jurisdiction and are stricter for outbound. Check them properly before deploying, especially for cold outbound.

Cost and build reality

Per-call running costs are low. The build and the ongoing supervision are where the real commitment sits, and a deployment without supervision degrades.

Frequently asked questions