A text agent that answers badly produces an unread message. A voice agent that answers badly is a person on a phone, in real time, forming an opinion about your company. That asymmetry is why we test outbound voice differently from everything else we ship.
The latency budget comes first
Before any accuracy work, fix the clock. End of caller speech to first audible token should sit under roughly 800 ms; any gap heading past 1.5 seconds needs a filler utterance or a handover, because callers interpret silence as a dead line. Measure the whole chain separately — endpointing, transcription, model, synthesis, network — because the component that blows the budget is rarely the one people blame.
The case library
Build a fixed library of recorded and synthesised calls and replay it on every prompt, model or voice change. Ours covers, at minimum:
- Accents and dialects across the markets the agent will dial, including non-native speakers of the call language.
- Noise floors — street, car cabin, open-plan office, hands-free with echo.
- Barge-in — the caller talking over the agent mid-sentence, twice in a row.
- Silence and hesitation — three seconds of nothing; "uhh… hold on".
- Off-script intent — the caller asking something the agent has no tool for.
- Hostility and abuse — the agent must de-escalate or exit cleanly, never argue.
- Compliance triggers — a recording objection, a request to be removed, a mention of a data-protection right. Each must route deterministically, not probabilistically.
- Wrong person, voicemail, IVR trees — outbound-specific and routinely forgotten.
Each case carries an expected outcome and a scoring rule, so a change produces a diff rather than a vibe.
Scoring what actually matters
Word error rate is a component metric, not a product metric. Score at the call level: did it reach the intended outcome, did it hand over when it should have, did it say anything it was not allowed to say, how long did it take, and what did it cost. A run that improves transcription accuracy while pushing handovers down is usually a regression, not a win — the agent has learned to bluff.
Detecting a synthetic caller
Outbound systems increasingly reach other machines. Combine signal cues — unnaturally clean spectra, absent breath and room tone, uniform pacing — with behavioural cues — instant answers to open questions, no barge-in, perfectly repeated phrasing. Emit a confidence score, and use it to route rather than to judge: log it, shorten the script, hand to a human on ambiguity. A hard block on a false positive is a customer you hung up on.
Guardrails you test explicitly
The escalation path is a test case, not a fallback. Write down before launch what "I do not know" sounds like, when the agent must stop, and what state the human receives — full transcript, caller identity, what was already promised. Then include broken handovers in the regression suite, because that is the failure that produces complaints.
Then dial, slowly
Shadow mode first: the agent runs on recorded traffic, nobody hears it, you compare its calls with what a human did. Then a small live cohort with every call reviewed. Then widen. Keep the manual path working for the entire rollout.
The architecture behind this — tool boundaries, escalation, evaluation loops — is described in agentic AI in production, and the full product picture including a voice pilot scope is on the Agentic AI page.
Frequently asked
What is outbound voice testing?
Systematically exercising an AI voice agent against recorded and simulated calls — accents, noise, interruptions, silence, hostile responses — with measured latency and a scored transcript, before it dials real numbers.
What latency does a voice agent need?
Aim for under roughly 800 ms from end of user speech to first audible response, and never let a silent gap exceed about 1.5 seconds without a filler or a handover. Callers read silence as a dropped line long before they read it as thinking.
How do you detect a synthetic caller on the other end?
Combine signal cues (unnaturally clean spectra, absent breath and room tone, suspiciously uniform pacing) with behavioural cues (instant responses to open questions, no barge-in). Treat it as a confidence score that routes the call to a person, never as a hard verdict.