Orca Security cuts natural language query costs 90% with Amazon Bedrock

Orca Security and AWS have built Discovery AI, a natural language agent on Amazon Bedrock that turns plain-text questions into valid Sonar queries. According to AWS, Orca reduced query costs by 90 percent, input tokens by 93 percent, and latency by 75 percent compared to its initial baseline.
- Orca Security and AWS built Discovery AI, which converts natural language questions into Sonar queries on Amazon Bedrock.
- According to AWS, query costs fell 90 percent, input tokens 93 percent, and latency 75 percent versus the initial baseline.
- The initial prototype used roughly 2 million input tokens and about 60 seconds per workload; the production system responds in 15 seconds.
- A nine-stage pipeline uses Claude Haiku 4.5, Amazon Nova 2 Lite, Amazon Titan embeddings, and Cohere Rerank 3.5.
- Example retrieval was the biggest accuracy lever, according to AWS.
The problem: a query language with 900+ models
Orca Security provides context-aware security for cloud, code, and AI. Its Sonar system is a domain-specific query tool where users search, filter, and explore across the entire data graph. According to AWS, Sonar covers more than 900 cloud models, meaning resource types such as an Amazon EC2 instance or an Amazon S3 bucket.
Querying that data means building JSON objects with exact model names, choosing from 18 operators (comparison functions such as equals or contains), and navigating parent-child hierarchies across cloud providers. Doing this well requires deep familiarity with the Sonar schema, which is a barrier for analysts who need fast answers.
Orca's first proof of concept loaded the full Sonar schema into a single prompt on the largest available model, including over 900 model definitions, every relationship type, and many example queries. Design partners and internal users responded positively to the output quality. At production scale, however, three drawbacks emerged:
- Token use was roughly 2 million input tokens per workload, which drove up cost.
- End-to-end response times were around 60 seconds per workload.
- Accuracy on complex, multi-model queries was limited, because nested relationships and multi-condition queries often needed refinement.
A nine-stage pipeline with different models per stage
Orca designed Discovery AI as a hybrid on Amazon Bedrock, combining Retrieval Augmented Generation (RAG) with schema-aware query construction. According to AWS, the team chose Bedrock because it offers multi-model access through a single API, so each stage can use a different foundation model without separate integrations. It also inherits AWS governance controls through IAM, VPC endpoints, and AWS CloudTrail, which met Orca's security requirements. The model-agnostic design lets the team swap in newer model versions without rearchitecting the pipeline.
The pipeline runs in nine steps:
- Request entry and context setup: the request is combined with the user's organization, connected cloud providers, and metadata.
- Quick deterministic matcher: regex and rule-based matching detects structured inputs such as IP addresses, CVEs, ARNs, file hashes, and alert IDs. A match returns a predefined filter through a fast path without LLM inference.
- LLM guardrail screening: blocks prompt injection, off-topic requests, and unsupported inputs.
- Embedding-based candidate recall: Amazon Titan Text Embeddings retrieve the 20 nearest previously validated examples.
- Neural relevance reranking: Cohere Rerank 3.5 on Amazon Bedrock selects the five highest-signal examples.
- Relevant model selection: more than 900 Sonar models are narrowed to a small set, supplied with schemas ranging from compact micro schemas to more detailed mini schemas.
- Agentic filter generation: a tool-augmented agent builds the filter and can refine it iteratively.
- Static and API-backed validation: the filter is checked statically and then against Orca's Sonar API. Errors go back to the generator with the offending element identified, and if the maximum number of retries is reached, an invalid-filter response is returned.
- Post-processing and response: the final response is prepared.
Which model does what
The team matched models to each stage's needs. Anthropic Claude Haiku 4.5 powers the Model Selector and Generator, which require strong reasoning for accurate query construction. Amazon Nova 2 Lite handles the Guardrail and Validator, where fast inference at lower cost is prioritized. Amazon Titan provides embeddings for example retrieval, and Cohere Rerank 3.5 reranks the candidates.
AWS says example retrieval was the single biggest accuracy lever in the pipeline. Strong examples teach the Generator how to translate questions into Sonar queries more efficiently than the schema alone. The Orca team benchmarked several retrieval strategies on Bedrock, and according to the post, pure top-K semantic search on Titan embeddings was fast but tended to return near-duplicate examples.
What it means
According to AWS, the production system handles thousands of queries per day at 15-second response times, down from around 60 seconds in the proof of concept. For security analysts, that means asking questions about cloud infrastructure in plain language rather than learning the Sonar schema. The case also shows a pattern of splitting a task across several smaller, specialized models and deterministic shortcuts instead of sending everything to the largest model in one prompt.


