Blocker 37 · AI

Your voice bot sounds like a robot: natural conversation with Amazon Connect agentic voice

A voice bot can give the right answer and still lose the caller. It interrupts, waits too long, mispronounces a code or reads out an asterisk. Amazon Connect agentic voice gives you control over how the AI agent hears, when it speaks and how it sounds.

640 msthe default end-of-turn silence timeout for Advanced ASR. It is the fallback: the confidence threshold is the primary signal that the caller has finished.Source: AWS, agentic voice best practices documentation

Illustration. The blocker: The voice bot sounds robotic. The fix: Natural voice that lets callers interrupt.

The blocker

The caller says “I need to change my—” and the bot starts talking. The caller reads out a policy number in two groups, pauses to breathe, and the bot decides they are done. The bot answers in a flat voice, spells a confirmation code as a word, and says “asterisk asterisk” because the text had markdown in it. After a long lookup, there are five seconds of dead air.

None of these are about the AI getting the answer wrong. They are about the conversation. But to the caller, the result is the same: it feels like talking to a machine, so they press zero or hang up. Containment falls, and the business decides voice AI “doesn’t work for our customers”.

Why it happens

  • Turn taking is based on silence alone. A fixed silence window either cuts callers off or makes the bot feel slow.
  • One setting for every moment. The same timing is used for a yes or no answer and for a 12-digit account number.
  • Barge-in is all or nothing. Either callers cannot interrupt anywhere, or they can talk over a legal disclosure.
  • Text written for screens is spoken aloud. Bullet points, brackets and raw IDs sound wrong through text-to-speech.
  • Silence during back-end calls. Nothing is said while the AI agent waits for a system to respond.

How Amazon Connect solves it

Amazon Connect agentic voice combines enhanced speech recognition with expressive voices. AWS states it is the default voice provider for Amazon Connect, with voices across more than 50 locales. The best practices guide is the main reference for everything below.

1. Hear the caller with Advanced ASR

In the bot’s speech configuration, set the provider to Amazon Connect agentic voice and the speech model preference to Advanced, as the configuration guide shows. Advanced ASR is a streaming recognizer. Instead of relying only on silence, it predicts when the caller has finished.

A turn ends when either condition is met first:

  • End-of-turn confidence threshold, default 0.7, range 0.5 to 0.9. This is the main lever.
  • End-of-turn silence timeout, default 640 ms, range 500 to 10,000 ms. This is the fallback.

You set both with Amazon Lex V2 session attributes, scoped to an intent and slot. AWS recommends defaults for normal conversation, and a threshold of 0.9 with a 6,000 to 8,000 ms timeout while the caller dictates an account number or one-time code. Reset them for the next slot. Do not apply one aggressive default to the whole bot to fix one slot.

2. Let callers interrupt, except where they must listen

Barge-in is on by default, which suits normal conversation. Turn it off only for prompts that must be heard in full, such as a recording disclosure or a confirmation readback, by setting the allow-interrupt attribute to false for that intent. AWS points out a common mistake: when the bot re-prompts after a caller goes quiet, that is end-of-turn detection, not barge-in. Fix it with end-of-turn settings.

3. Write for the ear, not the screen

Choose the voice in the Set voice block. Pick a language, choose a voice and listen to the sample before you publish. The engine reads common formats, such as phone numbers, dates and percentages, naturally.

The AI agent’s text still has to be speakable. AWS provides a speech output section to paste into the system prompt. It asks for full sentences, no markdown, lists, brackets or emoji, and short replies. For the cases the defaults miss, use speech control tags:

  • <spell> to read codes character by character.
  • <break> for a pause of a set length.
  • <speed> to slow down important or legal language.

Emotion tags exist but AWS marks them as beta. Malformed tags are spoken aloud, so validate them.

4. Never leave the caller in dead air

When a tool call takes time, AWS suggests finishing the work before opening the microphone, playing a short holding message, or enabling filler audio. The prompt engineering best practices add a similar instruction: send a short acknowledgment before a slow tool call.

5. Consider speech-to-speech with Amazon Nova Sonic

You can also configure Amazon Nova Sonic as a speech-to-speech model for a bot locale. It turns caller speech directly into expressive speech responses, while Amazon Connect still manages the flow. You then pick a Nova Sonic compatible voice with the generative speaking style in Set voice. When AWS announced it in November 2025, the release notes said Nova Sonic support was in US East (N. Virginia) and US West (Oregon), fully available in English and Spanish, and in preview for French, Italian and German. The configuration page lists Nova Sonic compatible voices for English and Spanish only. Check current Region and language support before you plan on it.

What good looks like

Callers speak normally and the AI agent waits while they read out a number, then answers quickly when they say “yes”. They can cut in to correct it, but they hear the required disclosure in full. Codes are spelled, amounts sound natural, and there is never silence while a system responds. When something sounds wrong, the team changes a session attribute, a prompt line or a tag, and tests again, without rebuilding the experience.

Where to start

  1. Listen to 20 recorded bot calls and note every cut-off, talk-over, pause and mispronunciation.
  2. Switch the bot to Advanced ASR and the flow to an agentic voice, and add the speech output section to the system prompt.
  3. Tune end-of-turn settings for the slots where callers dictate numbers, and turn barge-in off only for disclosures.
  4. Add holding messages around slow tool calls, then listen to a new set of calls and compare.
High-level architectureHow an AI agent hears, thinks and speaks on a call
Caller
  • Phone call
Amazon Connect flow
  • Set voice
  • Get customer input
Hearing
  • Advanced ASR
  • End-of-turn
  • Barge-in
Thinking
  • Orchestrator AI agent
  • Voice system prompt
Speaking
  • Agentic voice
  • Speech control tags
  • Nova Sonic option
  • People
  • Amazon Connect
  1. The flow runs a Set voice block that picks the Amazon Connect agentic voice provider, a language and a voice. A Get customer input block hands the call to the bot.
  2. The bot uses Advanced ASR, a streaming recognizer that predicts when the caller has finished instead of waiting only for silence.
  3. Session attributes tune end-of-turn detection and barge-in per intent and slot, for example more patience while the caller reads an account number.
  4. The orchestrator AI agent answers with a system prompt written for speech: full sentences, no markdown, codes wrapped in spell tags.
  5. The voice engine speaks the reply, with speech control tags for pace, pauses and spelling. As an alternative, a bot locale can use Amazon Nova Sonic speech-to-speech.
References

Official AWS documentation

Every Amazon Connect capability in this post links to AWS’s own documentation, so you can check the details for your Region and account.

  1. Agentic voicedocs.aws.amazon.com
  2. Agentic voice configuration guidedocs.aws.amazon.com
  3. Agentic voice best practicesdocs.aws.amazon.com
  4. Configure Amazon Nova Sonic speech-to-speechdocs.aws.amazon.com
  5. Flow block: Set voicedocs.aws.amazon.com
  6. Prompt engineering best practices for AI agentsdocs.aws.amazon.com
  7. Release notes for Amazon Connectdocs.aws.amazon.com
Start here

Tell us what you want to improve.

We will tell you whether a voice agent fits it, what it would take, and what it should return. You will hear back within a week.

  • AWS Advanced Tier Services Partner
  • AWS Migration Services Competency
  • Amazon Connect Delivery Partner
  • AWS AI Services Competency
  • AWS Experience-Based Acceleration (EBA) partner
  • ISO/IEC 27001 certified
  • SOC 2 Type II
Certified supplierMWDBE certifiedTexas HUB certifiedNCTRCA certified
+1 (214) 550-14936275 West Plano Parkway, Suite 500, Plano, Texas 75093