From keyword search to an AI assistant that finds the answer
A global hotel company with thousands of hotels and millions of guest interactions a year answered chat questions with Amazon Kendra. Guests got document links and snippets instead of answers, results changed with phrasing, and every new topic meant more synonym and relevance tuning. We replaced it with generative AI on Amazon Bedrock. Claude models search the company’s own knowledge by meaning, and AI tools identify the hotel and look up its amenities. Everything answers inside Amazon Connect chat, with a live agent one step away.
- 88–92%
- first answers correct, up from ~70%
- 15–20%
- agent-efficiency improvement
- 9 months
- from discovery to full rollout
- Zero
- manual synonym tuning
- Guest chat search moved from Amazon Kendra to generative AI on Amazon Bedrock.
- The first AI answer is right 88–92% of the time, up from about 70%.
- Agents identify the hotel and look up amenities; anything uncertain goes to a live agent with the full conversation.
- Guests get answers instead of links: first-response accuracy up from ~70% to 88–92%.
- 15–20% more agent capacity from the same team.
- A foundation for voice bots, agent assist and multilingual chat.
- Agents only see the chats the assistant could not finish, with the whole conversation.
- An admin console to manage prompts, agents and feature flags.
- Rolled out in phases, with feature flags for fast rollback.
- RAG on Amazon Bedrock (Claude) with embeddings, semantic retrieval and tool calling; Lambda orchestration behind Amazon Connect chat.
- Prompt versioning, guardrails, confidence thresholds and AI-specific metrics in CloudWatch.
- Two Availability Zones, private subnets, KMS, WAF and CloudFront; Terraform and Bamboo CI/CD.
One chat, from both sides
On the left, the guest’s chat. On the right, what the AI does: it works out the hotel, retrieves the right knowledge, writes the answer on Amazon Bedrock, or hands over to a person. Pick a question to see each path.
- Question
- Which hotel?
- Search knowledge
- Write answer
- Done
- Hotel
- —
- Topic
- —
- How sure it is
- —
- Waiting for a question
- Messages
- 0
- Sources used
- 0
- Lookups
- 0
- Agent needed
- …
Illustrative chats. The hotel, guests and amenities are fictional; the flow follows the production design: a hotel identification agent, semantic retrieval, answers generated on Amazon Bedrock, and a live agent when confidence is low.
The challenge
Guests ask about reservations, amenities, loyalty and policies, and they expect an answer, not a list of documents. Kendra was good at finding documents, but it could not understand intent, summarize, or follow a conversation over several turns.
So guests got links and partial snippets, asked again, and ended up with a live agent. Answers depended on how a question was phrased. Every new use case, such as hotel-specific FAQs or a policy change, meant more manual tuning of indexes, synonyms and relevance rules.
The company wanted conversational answers, more chats resolved without an agent, less tuning, and a foundation for voice bots, agent assist and multilingual chat later.
What made this modernization different
Swapping one AWS service for another is easy. Getting a generative AI assistant to answer reliably, at enterprise scale, is the work.
- 88–92%
of first answers correct, up from about 70%.
Measured by reviewing samples of real production chats, against a target of 85%.
When the hotel isn’t clear, the assistant asks before it answers.
“What about breakfast?” depends on the hotel. The identification agent asks a clarifying question instead of guessing, and keeps the hotel for the rest of the chat.
Content cleaned up and organized before the new model went live.
Answers are only as good as the content. We set chunking and metadata standards and validate content before it is embedded.
Every AI change released gradually, and easy to switch off.
Prompts are versioned and released in phases behind feature flags and canaries, so a change can be rolled back in minutes.
Answer quality measured, not just whether the servers are up.
We log prompt versions and outcomes, and review transcripts regularly, so quality is measured, not assumed.
What changed
- AnswersBefore: Document links and partial snippetsAfter: A conversational answer, grounded in retrieved knowledge (RAG)
- First answer rightBefore: About 70% of the timeAfter: 88–92%, from sampled production chats
- ConversationBefore: Each message on its ownAfter: Multi-turn, with session context and history
- Which hotelBefore: Results for every hotelAfter: A hotel identification agent finds the right hotel, and asks if it is not clear
- TuningBefore: Manual synonyms, indexes and relevance rulesAfter: Semantic retrieval; prompts versioned and released behind feature flags
- KnowledgeBefore: Documents written for a search indexAfter: Chunked, tagged by topic and hotel, validated before embedding
- EscalationBefore: Repeat questions ended with an agentAfter: Low-confidence answers hand over to a live agent with the full conversation
- ControlBefore: Changes needed engineersAfter: An admin console for prompts, agents and feature flags, with role-based access
- SecurityBefore: Search with broad accessAfter: Least privilege, KMS encryption, TLS, private subnets, AWS WAF at the edge
- ObservabilityBefore: Infrastructure metrics onlyAfter: Prompt versions, response outcomes and AI quality metrics in CloudWatch
Hover or tap a tile to see what it replaced.
The right answer, the first time.
Two KPIs were agreed with the business: how often the first AI answer is right without rephrasing or escalation, and how many chats are resolved without an agent. Both were compared against the Kendra-based search.
- First answer correctBefore~70%After88–92%
- Agent efficiencyBeforeBaselineAfter+15–20%
- Relevance tuningBeforeManual, ongoingAfterPrompts and tools in code
First-response accuracy sampled and reviewed across production chat transcripts (target 85%). Agent-efficiency improvement as documented for the engagement.
How a guest question gets answered now
The guest asks in their own words. The assistant works out the hotel, finds the knowledge, writes the answer, and hands over to a person when it is not sure.
- 01 · AskGuest types a questionIn Amazon Connect chat. Lambda adds the session context and conversation so far.
- 02 · ContextWhich hotel?If the hotel is not clear, the hotel identification agent asks first instead of guessing.
- 03 · RetrieveSemantic searchEmbeddings find the most relevant knowledge chunks for that hotel.
- 04 · AnswerAmazon Bedrock writes itClaude generates a conversational answer from the retrieved knowledge, calling tools such as amenity lookup when needed.
- Answered in chat
- Live agent
- 05 · EscalateOnly if neededLow-confidence or unresolved chats go to a live agent, with the full conversation.
How we ran it
- 1Discover and designWorkshops with business, application, security and operations; target architecture; Dev and UAT environments; baseline KPIs.
- 2Core frameworkLambda orchestration, Amazon Connect chat integration, the first Bedrock calls, request and session schemas, logging and retries.
- 3Understand and retrieveQuery reformatting, chunking and metadata, the embedding pipeline, semantic retrieval, compared against the old keyword results.
- 4Optimize answersPrompt templates, guardrails on tone and length, confidence thresholds, fallback and clarification logic.
- 5Agentic AIThe hotel identification agent: when to ask, structured lookup tools, context kept across the conversation.
- 6Admin, security and readinessAdmin console with role-based access, IAM hardening, KMS and TLS checks, dashboards, runbooks and an operational readiness review.
- 7UAT and go-liveFormal UAT, deployment through CI/CD and infrastructure as code, phased traffic with feature flags and canaries, then hypercare.
The outcome
- Guest chat search moved from Amazon Kendra to generative AI on Amazon Bedrock
- First-response accuracy up from about 70% to 88–92%
- 15–20% improvement in agent efficiency
- Agentic tools for hotel identification and amenity lookup, with clarifying questions when context is missing
- Multi-turn conversations, with handoff to a live agent carrying the full history
- An admin console for prompts, agents and feature flags
- Phased production rollout with feature flags and canary releases
- No manual synonym or relevance tuning
- AI
- Amazon Bedrock
- Claude models
- Retrieval-augmented generation
- Tool calling
- Agents
- Hotel identification
- Amenity lookup
- Knowledge
- Embeddings
- Vector store (Amazon OpenSearch)
- Amazon S3 knowledge base
- Contact center
- Amazon Connect chat
- Live agent handoff
- Compute and edge
- AWS Lambda
- Amazon CloudFront
- AWS WAF
- Security and operations
- AWS IAM
- AWS KMS
- Amazon CloudWatch
- Dynatrace
- Delivery
- Terraform
- Bamboo CI/CD
- AWS CDK
Five lessons shaped how we build generative AI for contact centers now. Answers need prompt governance, or tone and length drift. Quality depends on the content and how it is chunked as much as on the model. Vague questions need a clarify-first agent, not a guess. AI needs its own observability. And it should reach customers in phases, behind feature flags.
A partner AWS has validated, on exactly this kind of work.
- AWS Advanced Tier Services PartnerRecognized by AWS for proven delivery, certified people and customer outcomes.
- AWS Migration Services CompetencyValidated by AWS for moving and modernizing workloads on AWS at scale.
- Amazon Connect Delivery PartnerValidated by AWS to deliver Amazon Connect, including migrations from Avaya, Genesys, Cisco and other platforms.
- AWS AI Services CompetencyValidated by AWS for building AI into real products, including the contact center.
- AWS Experience-Based Acceleration (EBA) partnerWe run EBA engagements with AWS to speed up migrations with hands-on, time-boxed sprints.
- ISO/IEC 27001 certifiedOur information security management system is independently certified to the international standard.
- SOC 2 Type IIIndependent auditors have tested our security controls over time, not just at a single point.
More work
All case studies →- Amazon Connect
- Amazon Lex
- AWS Lambda
- Amazon Connect
- Contact flows
- AWS Lambda
- Amazon EKS
- Amazon Kinesis
- Amazon DynamoDB








