NVIDIA Workload Map Lets Kubernetes Tools Read AI Workloads Without Custom Adapters

The adapter problem on Kubernetes

On Kubernetes, a real AI workload is rarely a single pod. According to NVIDIA, a distributed training job or a multi-node inference server is a small tree of resources: groups of pods, each with a role, a replica count, a pod template and a status that depends on the workload type.

Teams building schedulers, controllers, dashboards or cost tools must answer the same questions for every custom resource definition (CRD). What is the structure of the workload type? How is its status interpreted as a phase? How can the pod spec be changed without breaking the workload? The answers end up in a per-CRD adapter.

NVIDIA says this turns adapter development into a roadmap item for each framework, which is why support for a new workload type so often waits for a release. The NVIDIA Run:ai team faced the same challenge while building its platform, the company says.

The post illustrates the issue with a JobSet that has one leader Job and two worker Jobs, each pod requesting four GPUs, for a total of 12 GPUs. Because kubectl lists the JobSet, its Jobs and pods as separate resources, a platform needs a definition to interpret their roles and aggregate the requests. Before the JobSet definition is registered, NVIDIA Run:ai shows the component Jobs as three separate workloads, each requesting four GPUs.

How Workload Map works

Workload Map, formerly called Karta, ships as a Kubernetes CRD plus a Go library that interprets workload definitions. The CRD's resource kind remains Karta. Each definition specifies where to find pod templates, replica counts and status fields for its workload type.

With a definition in hand, a consumer can:

• Extract pod templates, replica counts, status and metadata.
• Update pod specs, labels and annotations across every instance.
• Understand the workload hierarchy, for example a JobSet's replicated Jobs and their pods.

KAI Scheduler uses the definitions to interpret pod grouping and gang-scheduling instructions, while NVIDIA Run:ai uses them for workload management and visibility. NVIDIA says each consumer acts on the same description, which reduces the need for separate framework-specific parsers.

Workload knowledge moves out of compiled adapters and into data that developers can write, review and apply to a cluster like any other manifest, according to the post.

The JobSet definition and the prebuilt catalog

NVIDIA walks through a JobSet definition. The root component declares the workload's kind and how its status is read. A statusMappings section acts as a translation layer: JobSet does not report a plain "Running" condition, so the definition derives it with a jq expression over replicatedJobsStatus and maps the native Completed and Failed conditions onto a shared vocabulary. A RayJob or a LeaderWorkerSet, whose status looks very different, maps to the same categories.

Child components describe the structure inside the workload. For JobSet, podTemplateSpecPath points to each Job's pod template, and scaleDefinition computes the number of pods per replicated Job as replicas times parallelism. An instanceIdPath takes each instance's name from the replicated Job name, which is how the leader and worker roles appear.

Workload Map includes a catalog of prebuilt, tested definitions for JobSet, PyTorchJob, RayCluster, RayJob, RayService, KServe, Knative, MPIJob, NVIDIA NIM, LeaderWorkerSet, Milvus, Grove PodCliqueSet and NVIDIA Dynamo. Further workload types can be added by writing a YAML definition that describes their structure and status.

What it means for platform teams

NVIDIA says a single JobSet definition supports workload inspection through the Workload Map CLI, gang scheduling via KAI Scheduler and unified management in NVIDIA Run:ai. Elotl's Nova also uses Workload Map to interpret workload structure and resource requirements for multi-cluster scheduling decisions.

For teams that run several frameworks, the approach means supporting a new workload type can involve writing a definition rather than shipping new adapter code. NVIDIA suggests users start by exploring the workload catalog for a framework they already use, and invites them to join the NVIDIA DSX AI Factory Slack workspace to discuss definitions and integrations.