deepset unveils open reference architecture for self-improving agents at GTC Berlin

Reference architecture for agents that improve under oversight

According to deepset, the open reference architecture was developed through its technical collaboration with NVIDIA. It combines Haystack orchestration with NVIDIA open models and software for optimization and governed execution. The company says it gives organizations a foundation for agents that can learn from feedback and propose improvements while keeping sensitive workloads on infrastructure they control, including locally controlled and air-gapped environments.

Malte Pietsch and Julian Risch presented the topic on the GTC stage in a session titled "Evolve Within Bounds: Secure, Self-Improving Agents for On-Prem AI." The session dealt with keeping agents effective once they are in production and their data, tools and operating environment change.

The accompanying cybersecurity demonstration showed one agent analyzing confidential source code for vulnerabilities while a second agent examined its runs and proposed improvements. Proposed changes are evaluated before promotion, and approval, audit and rollback controls keep people in charge, according to deepset.

For developers, deepset points to a related log-analysis agent walkthrough. It shows a Haystack agent running on NVIDIA DGX Spark analyzing traces, testing configuration changes and recommending improvements using local models.

Cost optimization and custom model services

The second announcement covers new agent cost-optimization capabilities and model services. deepset says they are meant to help government, defense and industry reduce AI operating costs while maintaining high quality.

Haystack Enterprise Platform connects development, execution, optimization and governance, with capabilities to compare operating costs, evaluate answer quality and trace problems. Custom model services complement this by helping organizations adapt specialized models for their applications.

NVIDIA also featured deepset in its roundup of European public sector projects using open models. According to deepset, the article highlights Haystack Enterprise Platform as the foundation of the European Commission's AI@EC platform and describes the platform expansion. It also covers a planned sovereign cloud offering from secunet, NVIDIA, Red Hat and deepset for organizations handling classified information.

House of Sovereignty, panels and a hack night

Alongside its conference presence, deepset opened its Berlin headquarters as a meeting place for people working on AI that organizations can trust, improve and operate on their own terms. A panel with Roberto Sgnaolin, Christian Knobel and Sven Giesselbach asked how organizations can combine sovereignty with fast innovation cycles, adopting new capabilities while keeping control over data, models, infrastructure and applications.

A community hack night let developers build applications with Haystack and work alongside the deepset team. On stage, Milos Rusic joined Andreas Simon Rueckriegel, Michael Frings and Alexander Heinrich to discuss AI in critical use cases and environments with demanding security requirements. The company argues that such conversations go beyond model capability: organizations also need to govern what an agent can access, understand how it reaches an outcome and decide where the complete application runs.

deepset says Jensen Huang and Martin Blessing, described as German Chancellor Friedrich Merz's personal envoy for increasing foreign investment in Germany, visited its booth to talk about open source and sovereign AI. Huang signed the NVIDIA DGX Spark that ran the live cybersecurity agent. The team also met Karsten Wildberger, Germany's Digital Minister, and discussed joint plans with partners including n8n, Materna, secunet and Schwarz Digits.

What it means for organizations

The announcements target organizations that want to use agents without giving up control over their models, data and infrastructure. The reference architecture is open, and the cost tools address the expense of running agents at high quality, a concern deepset names explicitly for government, defense and industry.

deepset frames its approach as helping teams build, run, improve and govern AI as a complete system. Its stated view is that sovereignty means having choices and being able to change them, and that it takes an ecosystem of open source developers, infrastructure providers and application builders to deliver it.