AWS shows how to deploy agentic retrieval on Bedrock with one CDK app

Where standard RAG falls short

AWS opens by describing where ordinary Retrieval Augmented Generation (RAG) breaks down. Most systems embed the question, fetch the nearest chunks from one index, pass them to a model and return the answer: one retrieval, one generation. AWS says that is the right design for a large class of questions.

It fails in three recognizable ways, according to the post. When an answer spans two corpora, a single retrieval returns half of what the model needs, and the answer comes back fluent and incomplete. When the right source depends on the question, a single index cannot choose. And when a question needs a second look at the corpus after the first result, there is no place for that to happen.

Agentic retrieval addresses all three by putting a reasoning agent in the loop. The agent decides whether to retrieve, which knowledge base to consult, and whether to go back for more before composing an answer.

What the Managed Knowledge Base changes

The post builds on the Managed Knowledge Base (Type: MANAGED) in Amazon Bedrock. AWS says Bedrock handles ingestion, storage, indexing and retrieval, including embedding and reranking with service-managed models by default, so there is no vector database to provision, scale or patch. The AgenticRetrieveStream API provides multi-turn planning, tool-driven retrieval and cited answers as a service feature.

AWS compares this with the customer-managed (DIY) option, where the user operates the vector store, for example Amazon OpenSearch Serverless:

• Agentic retrieval through AgenticRetrieveStream is supported on managed knowledge bases and not on customer-managed ones.
• AgentCore Gateway integration is likewise supported only on managed knowledge bases.
• The managed data store scales automatically, while a DIY store must be provisioned, scaled and maintained.
• Embedding and reranking are built-in managed models with no extra cost, versus models the user configures.

The customer-managed route remains the right choice, AWS says, when you want to reuse an existing OpenSearch Serverless collection or meet specific data-residency, tuning or compliance requirements.

The architecture also uses Amazon Bedrock AgentCore: a Gateway that exposes each knowledge base as a Model Context Protocol (MCP) tool, and a Runtime that hosts the reasoning agent.

Observability and evaluation built in

AWS argues that an agent which reasons and retrieves in a loop takes a different path on every question, and by default you can see neither that path nor whether the answer was any good. In its words, observability becomes the only way to know the system works.

The solution therefore ships two Amazon CloudWatch dashboards spanning seven layers of telemetry, plus two forms of evaluation, on-demand and continuous, all from the same deployment. AWS says roughly half of what the app provisions exists to show what the agent did. Cited answers trace back to source passages so responses stay auditable, and faithfulness and correctness scores let teams catch answers drifting from their sources before production.

Why CDK, and what it means for teams

The post follows an August 2024 article that deployed a single-shot RAG design as a CDK application with three stacks, an S3 corpus, an OpenSearch Serverless collection and index, a knowledge base and a data source. In the new design the vector store is gone, so there is no OpenSearch Serverless collection or OCU capacity to provision.

AWS says CDK fits because the four stacks are densely cross-referenced. The Gateway needs both knowledge base ids, the agent needs the Gateway URL and ARN, and the dashboards need the knowledge base ids, the Gateway ARN and the agent runtime id: seven values crossing stack boundaries. With raw templates, these would be kept in agreement by hand. In CDK, according to AWS, passing the construct derives both the export and the dependency edge, and CDK works out the deployment order.

For teams already running single-shot RAG, the post positions the managed knowledge base as the route to agentic retrieval, since AWS states the capability is available only there. Teams that need their own vector store would stay on the customer-managed option and would not get agentic retrieval or the Gateway connector.