Nine questions to ask a voice AI vendor before you sign

Why demos are a poor test

Every voice AI demo you will sit through this quarter works. It works because it runs in a browser tab, on a laptop, in a quiet room, with a good microphone, on a script the vendor chose. Nothing in that sentence describes a phone call.

The gap between the two is where pilots die: a caller on a moving train, on a mobile network that loses a fraction of a second of audio, switching between Hindi and English inside one sentence, talking over the answer because they already know what they want.

Nine questions follow. Each one has an answer that differs between vendors, and each has a wrong answer worth listening for. The failures the questions are designed to catch are collected in why voice AI agents fail.

What did the slowest one call in twenty look like last week?

Not the average. The average is the number that hides the calls where your customer hung up.

The tell: a vendor who cannot answer this does not record how long each caller waited, which means nobody can tell you afterwards why a bad call was bad. Timing per turn either exists or it does not, and it cannot be reconstructed later. There is more on why the published figures rarely compare in response time in a voice agent.

Where are your servers, physically?

Every leg of a voice call goes to a server somewhere and comes back, on every turn. If those servers are on another continent, that round trip is paid before any work has been done.

On our own systems, reaching speech recognition from Mumbai takes about 20 ms. Reaching the same models hosted in the United States takes about 300 ms. No engineering wins that back, and the old telephony guidance in ITU-T G.114 is a reminder of how little delay a conversation tolerates before it degrades.

The tell: “our cloud provider has a region in India” is not the same answer as “the speech, the reasoning and the voice are all served from India”. Ask about all three.

Can I interrupt it in the last two seconds of a long answer?

Try it in the demo. Many systems stop listening the moment they have handed the audio over, which is several seconds before the caller has finished hearing it. The result is an agent you can interrupt at the start of a sentence and not at the end.

The tell: the vendor demonstrates interruption by breaking in at the beginning. Ask to do it yourself, near the end.

What happens when a door slams?

The opposite failure. On an Indian mobile call there is always background noise. A system tuned in a quiet room stops mid-word every time a bus passes.

The tell: the demo is conducted somewhere silent. Ask them to take it outside. Both of these failure modes are covered in barge-in, and the more common complaint, that the voice AI agent interrupts callers, has three causes worth testing for separately.

Show me a sentence with two languages in it

Not one language and then the other. One sentence: Hindi, an English product name, back to Hindi. That is how your customers speak.

The tell: the answer is a language count. “We support 40 languages” means the system can be set to each of them, which is a different claim from handling two at once. The detail that matters, and the reason a single setting cannot work, is in callers who mix Hindi and English.

What are the rules for fetching a person, and who writes them?

The answer should be that you write them and they are enforced in advance: a subject the agent never touches, a value above a threshold, an upset caller, a caller who asks.

The tell: handover happens only after the agent has failed. That means every customer who needed a person spends ninety seconds discovering the agent could not help, and arrives at your colleague annoyed. See when a voice agent should fetch a person.

What does my colleague see when the call is transferred?

They should see who is calling, what was asked, and what has already been looked up. If the call is merely forwarded, your customer repeats themselves, which is the experience they rang to avoid.

Who bills the call minutes, and can I keep my numbers?

Minutes are often the largest line on the invoice and are frequently billed by a different company from the one selling you the software. Ask for both figures before comparing quotes, and ask what happens to your numbers if you leave. The two common arrangements are set out in SIP trunking. If you resell voice agents to your own clients, ask your supplier the same two questions; the page on our white label voice AI platform sets out what a reseller should expect.

What has gone wrong on a real call, and what did you change?

The last one, and the most revealing, because the honest answer is specific and the dishonest answer is smooth.

The tell: “nothing significant”. Every system that has been on a live phone line has a list. A vendor who cannot produce one has either not been on a phone line or is not watching. Ours starts with the day it took four attempts to complete one phone call, three of which found a defect the entire test suite had passed.

What to do with the answers

Pick one call type your team handles a lot and dislikes, and run the same two-week test with each shortlisted vendor on your own inbound line. Agree beforehand what it is measured against: calls answered outside working hours, callbacks made within a minute, appointments confirmed without a person. Then the comparison is between products rather than between sales decks.

Our own version of that test is described on the page about which calls to hand over first.