Response time in a voice agent, and how to ask about it

What the caller experiences

A caller does not experience a pipeline. They experience silence: the gap between the moment they stop talking and the moment they hear a voice. Everything that happens inside that gap is the vendor’s problem, and the caller’s only measurement is whether the pause felt normal.

Roughly, and consistently with how people react on ordinary telephone calls:

• Under about a second feels like talking to somebody.
• One to two seconds feels like talking to somebody who is distracted.
• Past two seconds and callers start saying “hello?” because they assume the line has gone.

The old telephony standard on this, ITU-T G.114, is about one-way transmission delay rather than about thinking time, but it points at the same human threshold: conversations degrade well before a second of delay.

Why published numbers rarely compare

Almost every vendor in this market publishes a sub-second figure, and almost none of them say what was measured. Four things vary, and each of them can move the number by more than the difference between two vendors:

• Where the clock starts. From the end of the caller’s speech, or from the moment the system decided the caller had finished? Those are different numbers, and the second one quietly excludes a real part of the wait.
• Where it stops. At the first byte of audio produced, or at the first sound the caller can hear? A carrier connection sits between those two.
• What it was measured on. A browser tab on a laptop with a good microphone, or a telephone call over a mobile network.
• Average or worst case. An average across a thousand calls tells you nothing about the one where a customer hung up.

The question that gets a straight answer

Ask this: for real phone calls last week, what did the slowest one turn in twenty look like?

A vendor who records timing per turn can answer immediately. A vendor who cannot will give you an average, which is the tell. This is why Voicely records how long the caller waited on every turn of every call and keeps it, rather than reporting a single figure: the number is only useful on the call that went badly.

What makes it slow

In rough order of how much each one costs:

• Distance. Every leg of the work goes to a server somewhere. If those servers are on another continent, the network round trip is paid on every turn, before any work is done. What that costs on an Indian phone line is measured in why voice AI built for English struggles on Indian calls.
• Waiting too long to decide the caller has finished. Deciding too early interrupts people; waiting too long adds dead air to every turn. The setting is the pause threshold, or end-of-turn detection. The right value is different per language, which is why a system tuned only on English gets it wrong for code-switching callers.
• Doing the work in sequence rather than as it arrives. Waiting for a complete answer before starting to speak adds the whole length of the answer to the pause.
• Looking things up. A slow lookup in your own system is a slow call, and it is the one part of this a vendor cannot fix for you.