An AI voice agent answers the phone, talks with the caller, and does something useful: books an appointment, answers a question, takes a message. It works well on a narrow set of calls and badly on others. Most disappointing deployments come from asking the agent to do both.
This is what we have learned from building and supporting phone agents, written without numbers because the right ones depend on your business.
#What works well
Voice agents do their best work on calls that are routine, predictable and low-risk:
- After-hours calls. A caller who would otherwise hit voicemail gets an answer, and you get a record of what they wanted.
- Booking and rescheduling. The goal is clear, the data needed is small (name, time, reason), and success is easy to check.
- Frequently asked questions. Hours, location, prices you publish, how to prepare for a visit. The agent should answer from material you wrote and approved, not improvise.
- Qualifying callers. A few consistent questions sort serious inquiries from wrong numbers and pass the useful ones along.
- Taking messages. Capture who called, why, and how to reach them, then deliver it in a form someone will actually read.
- Handing off. Collecting the basics, then transferring the call with context, so the caller does not repeat themselves.
#What does not work well
Be equally clear about the other side:
- Angry or distressed callers. Someone who is upset wants a person who can take responsibility. A polite script makes it worse.
- Ambiguous, multi-part requests. “I need to move my appointment, but also ask about my last bill, and is my sister covered?” People handle this naturally. Agents tend to drop a thread.
- Poor audio and difficult speech. Heavy accents combined with a noisy line are harder for speech recognition, and a misheard name or number can derail the call. Test with the lines and accents your callers really have.
- Legal or medical advice. The agent should not give it. It can say what it cannot help with and route the caller to someone qualified.
- Anything where a wrong answer is expensive. Refund promises, quotes with exceptions, safety questions, contract terms. If a wrong answer is costly, a person should give it.
#The hand-off is the core design decision
The most important part of a voice agent is how it gives the call to a person. Decide these before you decide anything about the voice:
- When does it hand off? Define triggers: the caller asks for a person, the agent fails to understand twice, the topic is on a restricted list, or the caller sounds upset.
- Who receives the call? A named person or team, during hours when they are reachable. Outside those hours, the agent takes a message and says honestly when to expect a reply.
- What does the person see? A short summary and the transcript so far, so the caller never repeats their story.
- What does the caller hear? A plain statement, such as “I’m going to connect you with someone who can help.” No stalling and no pretending.
A caller who can reach a human easily forgives a clumsy agent. A caller who gets trapped does not.
#How to test before and after launch
Listening beats guessing. Do this:
- Listen to real calls. Not demos. Real recordings, with real interruptions, background noise and half-finished sentences.
- Read the transcripts. Look for misheard names, wrong intents, and places where the agent sounded confident while being wrong.
- Write a test script of awkward calls. Interruptions, a caller who changes their mind, a request outside scope, a request for a person.
- Keep reviewing after launch. New situations show up that nobody planned for. Add the good examples to your tests.
For Teldash, the call-center portal we built, this review loop was a core feature: agencies and their clients can open live calls and transcripts in a branded dashboard. Whatever platform you choose, make sure someone can actually see what the agent said.
#Test each language separately
If you take calls in English and Spanish, test both on their own. Do not assume that good results in one carry over to the other. Check names, addresses, numbers, dates and the way people actually speak, including regional vocabulary and callers who switch languages mid-sentence. Review transcripts for each language, and write approved answers in each language rather than relying on translation of an English script.
#Where to start
Pick one call type with a clear goal and low risk, usually after-hours booking or FAQs, and keep a person one step away. Expand only after you have read the transcripts and are satisfied.
Our guide to choosing a voice agent provider lists what to evaluate, and if you want to talk through your call types, you can book a 30-minute call.
