NVIDIA Opens Up Run:ai, Releasing Full GPU Orchestration Platform as Open Source

What NVIDIA released

The release includes the complete NVIDIA Run:ai platform and four components that can also be adopted on their own: KAI Scheduler, GPU Fractioning Operator, KAI Resource Management and Workload Map. Together with additional platform code, these pieces form the integrated platform. Teams can adopt it as a whole, build their own products on it, or pick individual components to fit their requirements.

NVIDIA says Run:ai has helped organizations run AI workloads on Kubernetes at scale in recent years, by dynamically allocating GPU resources and managing workloads from data processing and training to inference. By open sourcing it, NVIDIA argues, every NVIDIA DSX ecosystem partner and AI cloud can increase workload throughput, improve GPU utilization and run shared AI infrastructure more efficiently.

How to get it

The full source code is hosted in the NVIDIA Run:ai repository within the NVIDIA DSX AI Factory GitHub organization. Teams can inspect the code, contribute to it and build on top of it. The complete platform is also offered on NVIDIA NGC as a Community Product, with a Helm chart for installing the integrated stack.

Some conditions apply. The source code is governed by its applicable open source licenses. Downloading the Community Product from NGC requires acceptance of the applicable NVIDIA Software License Agreement and Product-Specific Terms for NVIDIA AI Products. No paid Run:ai subscription or NVIDIA support entitlement is required for the Community Product, but NVIDIA enterprise support is not included.

For NVIDIA Enterprise AI Factory deployments, Run:ai remains available through NVIDIA-supported commercial offerings, including as a standalone product, through NVIDIA Mission Control and NVIDIA AI Enterprise, and on select cloud marketplaces. NVIDIA notes that availability, licensing and support terms vary by offering.

KAI Scheduler and the modular components

KAI Scheduler is the Kubernetes-native scheduler that powers Run:ai. NVIDIA previously released it as open source under the Apache 2.0 license and contributed it to CNCF as a Sandbox project. According to NVIDIA, it remains the upstream engine behind Run:ai, and all scheduler development happens in the community repository.

The scheduler is built for GPU workloads at scale and covers:

• Gang scheduling with prescheduling in-memory simulation, so multipod jobs are placed all-or-nothing and are not evicted needlessly
• Fair-share, including time-based fair-share, and quota allocation across teams
• Topology awareness and bin-packing at the scheduling-decision level
• GPU sharing across single or multiple GPUs (MIG, MPS, time sharing, HaMi, CUDA 13.4 MPS fractioning)
• Dynamic Resource Allocation (DRA) support

NVIDIA says KAI Scheduler can be deployed in an existing cluster alongside the current software stack without the complete Run:ai platform. The GPU Fractioning Operator is described as enabling CUDA-native GPU fractions on Kubernetes clusters. KAI Resource Management handles organizational quota and hierarchy, and Workload Map provides a consistent workload-management contract.

Partner integrations

Eight NVIDIA DSX ecosystem partners are integrating the open source platform into their offerings:

• Crusoe integrates Run:ai with Crusoe Managed Kubernetes.
• Mirantis extends k0rdent AI with Run:ai for a lifecycle-managed, Metal-to-Model AI factory stack.
• Nebius offers Run:ai as a one-click application deployment in Nebius AI Cloud, on Nebius Managed Service for Kubernetes.
• OpenNebula Systems integrates it with the OpenNebula AI Cloud Platform through OneKS, for open and sovereign AI clouds.
• Rafay is expanding its integration to deliver Run:ai as a fully managed service, so AI cloud providers and enterprises can offer it through their own self-service catalogs.
• Saturn Cloud uses Run:ai to help AI cloud operators launch white-label AI inference services through its Token Factory platform, with per-token metering and automated billing.
• Spectro Cloud integrates it into PaletteAI, deployable through PaletteAI Studio.
• vCluster provides Run:ai through pre-validated deployment templates that stand up a complete environment per tenant, while Run:ai keeps scheduling across one GPU fleet.

NVIDIA also points to two related open source projects, Model Streamer and Grove, which it says accelerate model loading and orchestrate multicomponent inference workloads.

What it means for platform teams

For organizations that run shared GPU clusters, the release means the orchestration code behind Run:ai can be inspected, modified and extended rather than consumed only as a commercial product. Teams that already have a platform can fill a specific gap by adopting a single component, such as the scheduler or the fractioning operator. Cloud operators and software vendors in the DSX ecosystem gain a common base they can build into their own AI factory offerings. Those who need enterprise support will still have to use NVIDIA's commercial offerings, since the Community Product does not include it.