The blocker
The caller says “I need to change my—” and the bot starts talking. The caller reads out a policy number in two groups, pauses to breathe, and the bot decides they are done. The bot answers in a flat voice, spells a confirmation code as a word, and says “asterisk asterisk” because the text had markdown in it. After a long lookup, there are five seconds of dead air.
None of these are about the AI getting the answer wrong. They are about the conversation. But to the caller, the result is the same: it feels like talking to a machine, so they press zero or hang up. Containment falls, and the business decides voice AI “doesn’t work for our customers”.
Why it happens
- Turn taking is based on silence alone. A fixed silence window either cuts callers off or makes the bot feel slow.
- One setting for every moment. The same timing is used for a yes or no answer and for a 12-digit account number.
- Barge-in is all or nothing. Either callers cannot interrupt anywhere, or they can talk over a legal disclosure.
- Text written for screens is spoken aloud. Bullet points, brackets and raw IDs sound wrong through text-to-speech.
- Silence during back-end calls. Nothing is said while the AI agent waits for a system to respond.
How Amazon Connect solves it
Amazon Connect agentic voice combines enhanced speech recognition with expressive voices. AWS states it is the default voice provider for Amazon Connect, with voices across more than 50 locales. The best practices guide is the main reference for everything below.
1. Hear the caller with Advanced ASR
In the bot’s speech configuration, set the provider to Amazon Connect agentic voice and the speech model preference to Advanced, as the configuration guide shows. Advanced ASR is a streaming recognizer. Instead of relying only on silence, it predicts when the caller has finished.
A turn ends when either condition is met first:
- End-of-turn confidence threshold, default 0.7, range 0.5 to 0.9. This is the main lever.
- End-of-turn silence timeout, default 640 ms, range 500 to 10,000 ms. This is the fallback.
You set both with Amazon Lex V2 session attributes, scoped to an intent and slot. AWS recommends defaults for normal conversation, and a threshold of 0.9 with a 6,000 to 8,000 ms timeout while the caller dictates an account number or one-time code. Reset them for the next slot. Do not apply one aggressive default to the whole bot to fix one slot.
2. Let callers interrupt, except where they must listen
Barge-in is on by default, which suits normal conversation. Turn it off only for prompts that must be heard in full, such as a recording disclosure or a confirmation readback, by setting the allow-interrupt attribute to false for that intent. AWS points out a common mistake: when the bot re-prompts after a caller goes quiet, that is end-of-turn detection, not barge-in. Fix it with end-of-turn settings.
3. Write for the ear, not the screen
Choose the voice in the Set voice block. Pick a language, choose a voice and listen to the sample before you publish. The engine reads common formats, such as phone numbers, dates and percentages, naturally.
The AI agent’s text still has to be speakable. AWS provides a speech output section to paste into the system prompt. It asks for full sentences, no markdown, lists, brackets or emoji, and short replies. For the cases the defaults miss, use speech control tags:
<spell>to read codes character by character.<break>for a pause of a set length.<speed>to slow down important or legal language.
Emotion tags exist but AWS marks them as beta. Malformed tags are spoken aloud, so validate them.
4. Never leave the caller in dead air
When a tool call takes time, AWS suggests finishing the work before opening the microphone, playing a short holding message, or enabling filler audio. The prompt engineering best practices add a similar instruction: send a short acknowledgment before a slow tool call.
5. Consider speech-to-speech with Amazon Nova Sonic
You can also configure Amazon Nova Sonic as a speech-to-speech model for a bot locale. It turns caller speech directly into expressive speech responses, while Amazon Connect still manages the flow. You then pick a Nova Sonic compatible voice with the generative speaking style in Set voice. When AWS announced it in November 2025, the release notes said Nova Sonic support was in US East (N. Virginia) and US West (Oregon), fully available in English and Spanish, and in preview for French, Italian and German. The configuration page lists Nova Sonic compatible voices for English and Spanish only. Check current Region and language support before you plan on it.
What good looks like
Callers speak normally and the AI agent waits while they read out a number, then answers quickly when they say “yes”. They can cut in to correct it, but they hear the required disclosure in full. Codes are spelled, amounts sound natural, and there is never silence while a system responds. When something sounds wrong, the team changes a session attribute, a prompt line or a tag, and tests again, without rebuilding the experience.
Where to start
- Listen to 20 recorded bot calls and note every cut-off, talk-over, pause and mispronunciation.
- Switch the bot to Advanced ASR and the flow to an agentic voice, and add the speech output section to the system prompt.
- Tune end-of-turn settings for the slots where callers dictate numbers, and turn barge-in off only for disclosures.
- Add holding messages around slow tool calls, then listen to a new set of calls and compare.










