deepset expands Haystack Enterprise Platform to track and cut AI agent costs

A single platform for building, running, improving and governing agents

deepset announced the expansion at NVIDIA GTC Berlin 2026. The company says the platform now combines four connected areas: development and runtime as the foundation, production traces and cost data that feed improvements back into the agent harness, and governance controls across the lifecycle.

The first area covers the agent harness and context. deepset argues that an agent's behavior depends on more than its underlying model. The context it receives, the tools it can access, its memory, and the instructions and guardrails all shape the result. Built on open-source Haystack, the platform lets teams design and test workflows, connect models with enterprise data and tools, and inspect the components behind agent behavior. Its modular architecture is meant to allow swapping models, retrieval strategies and other components without rebuilding an application.

The second area is runtime and services. Teams can publish version-controlled agents through stable service endpoints, so applications can keep calling the same endpoint while the underlying agent is updated. Serverless execution is intended to reduce the need to provision dedicated infrastructure for each workload and let resources scale with demand.

Tracing and cost reporting to find expensive steps

The optimization loop is the centerpiece of the announcement. deepset says that choosing a model is only one part of managing AI costs, because the design and orchestration of an agent also determine how much inference it needs and whether it delivers the required results.

Production traces and cost reports are meant to help teams find expensive or unreliable steps in a workflow. Teams can inspect individual runs and spending across models, agents, services and workspaces. The company lists questions the tools are meant to answer:

• Which agents or models account for the most AI spending?
• Where are errors, unexpected behaviors or unnecessary execution steps occurring?
• Could changes to context, retrieval, model selection or tool usage improve efficiency?
• How can teams assess whether an optimization maintains the required quality?

The last question points to what is still ahead. Evaluations and more automated optimization recommendations are planned, building on the production visibility available today. Planned capabilities will help teams test model and harness changes against cost and quality objectives. Over time, deepset says, the platform will recommend improvements based on production evidence, with human approval before deployment.

Governance, deployment options and model services

For organizations running several agents, the platform offers shared governance capabilities, including role-based access controls, workspace management and operational visibility. A control plane and gateway architecture is designed to make agents, tools and skills accessible through standard interfaces, with common controls for permissions, policies and usage.

The production capabilities in the expansion are production tracing, cost reporting by model, agent, service and workspace (including alongside individual traces), production services, serverless deployments and flexible deployment. On deployment, the platform supports public cloud, European sovereign cloud environments such as STACKIT, and self-hosted infrastructure. deepset says this, together with Haystack's open-source, model-agnostic architecture, preserves flexibility across models and infrastructure, which it describes as essential for organizations with strict data, security or regulatory requirements.

Alongside the platform, deepset introduced custom AI model services for government, defense and regulated industries. Its Forward Deployed Engineers help organizations select, adapt and post-train decision models and small language models for specific tasks. These can work alongside LLMs in Haystack Enterprise Platform, and customers own the models developed for them.

What it means for teams running agents

For organizations that already run agents in production, the update puts cost and behavior data in the same place where agents are built and deployed. Teams can see which models, agents or workspaces drive spending and inspect individual runs without leaving the platform, according to deepset.

The more automated parts, namely evaluations and recommended improvements, are not yet available. deepset describes its goal as an operating system for sovereign AI agents, a common foundation to build, run, improve and govern agents, with more systematic evaluation and optimization ahead.