NVIDIA Donates Agent Substrate Runtime to CNCF Sandbox

A different approach to running agents

NVIDIA says AI agents spend much of their time waiting for a human prompt, a tool result or a model response. Keeping one Kubernetes Pod active for every session can leave compute idle, while creating Pods between turns introduces cold starts and loses the agent's state.

Agent Substrate creates a pool of ready Kubernetes worker Pods and uses a separate control plane to map many agents and their environments onto that pool. When an actor becomes idle, its sandbox is snapshotted, including memory and filesystem state, and the worker is released. When the actor is needed again, it is restored onto any available worker without a new trip through the Kubernetes API server, scheduler or kubelet.

NVIDIA describes the runtime as security-hardened and zero-trust. It supports sandboxed environments based on gVisor or microVMs, while the worker Pods remain ordinary Kubernetes Pods subject to cluster scheduling, quotas and placement policies.

Three workload types

The project is aimed at coding assistants, reinforcement learning post-training and short-lived GPU research sandboxes.

Coding assistants alternate between model calls and actions such as cloning repositories, editing files, running builds and executing tests. Their environments must persist between turns, but a service serving thousands of sessions cannot afford a dedicated warm sandbox for every user.

In reinforcement learning post-training, GPU trainers and generation servers work alongside CPU-heavy environments such as code sandboxes, compilers, browsers and simulators. NVIDIA says demand can arrive in bursts, and slow environment startup can leave GPUs idle while trainers wait for the next batch.

Researchers can also use GPU sandboxes to compile CUDA code, benchmark kernels or run numerical calculations. These sandboxes need to start quickly and share a limited GPU pool across an organization.

Integration with NVIDIA's AI infrastructure

Agent Substrate works with NVIDIA Dynamo and KAI Scheduler in the NVIDIA DSX AI Factory Platform. Dynamo coordinates inference across GPU workers, while Agent Substrate manages the agent environments.

Dynamo's router uses session identity, the location of the key-value cache and tool-call boundaries to keep related model calls on workers that already hold reusable context. It also exposes signals such as projected time to first token and GPU-pool saturation. Agent Substrate uses those signals to decide whether an actor should remain resident, be suspended during a long inference wait or resume as a response nears completion.

KAI Scheduler places Substrate worker Pods alongside GPU trainers and inference servers. NVIDIA says Agent Substrate is an independent CNCF Sandbox project that integrates with DSX OS components.

Upstream work and what it means for users

Recent and ongoing work includes request parking, which queues a request when all workers are busy instead of immediately returning an error. The project is also developing device support through Kubernetes Dynamic Resource Allocation, with NVIDIA GPUs planned as the first supported device. Other work adds per-container CPU and memory limits for microVM sandboxes, allowing an over-consuming container to be stopped while the rest of the sandbox continues running.

For users and platform teams, the project offers a way to preserve agent environments while sharing a smaller pool of compute workers. Its CNCF Sandbox status also places the runtime's development in an open-source Kubernetes ecosystem, while its integration with Dynamo and KAI Scheduler connects agent execution with inference and cluster scheduling.