CoreWeave Details How It Brings Up Multi-Rack NVIDIA Vera Rubin NVL72 Clusters

From one rack to a distributed system

CoreWeave describes a single NVIDIA Vera Rubin NVL72 rack as a co-designed system with 72 NVIDIA Rubin GPUs, 36 Vera CPUs, the NVIDIA NVLink 6 scale-up fabric, ConnectX-9 SuperNICs, BlueField-4 DPUs, high-speed storage, power and 45°C liquid cooling.

Connecting several racks, the company says, changes the engineering problem. Hundreds of Rubin GPUs, multiple NVLink domains, thousands of network connections, cooling, power, firmware and software have to behave as one. CoreWeave lists the risks: a slow GPU can become a cluster-wide straggler, a marginal network link can hurt collective performance, a cooling anomaly can cause throttling or a complete shutdown, and a configuration inconsistency can be hard to diagnose once hundreds of GPUs communicate at the same time.

The company points to its earlier deployments of NVIDIA L40S, H200, GB200 NVL72, GB300 NVL72 and RTX PRO 6000 as the experience it applies to Vera Rubin.

Automation and validation before production

New infrastructure at CoreWeave enters an automated lifecycle. Once hardware is plugged in and the liquid cooling loop is provisioned, automation detects new node baseboard management controllers, verifies serial numbers, bootstraps credentials and moves nodes into a staging environment. There, the company verifies hardware, updates firmware, applies metadata and validates systems before they advance toward production.

For Vera Rubin, CoreWeave extended its Mission Control software with Racky, a rack-management control layer, and Valvey, its programmable approach to liquid-cooling control. They work alongside the Rack LifeCycle Controller and put power delivery, the cooling loop and environmental sensing on the same software-defined lifecycle as compute.

Validation is layered. At node level, CoreWeave runs hours of repeated GPU diagnostics, tests data movement between CPU and GPU, checks GPU-to-GPU connections, and runs compute-intensive and realistic training workloads. At rack level, it runs standard and custom benchmarks, tests NVLink communication, and runs tightly synchronized jobs across all 72 GPUs. Racks that perform below the expected range go to troubleshooting rather than production. The final stage runs distributed workloads across multiple racks and forces traffic over the backend network.

A two-tier, non-blocking RoCE fabric

Between racks, distributed workloads rely on the backend network. CoreWeave says that if network capacity does not scale with GPU capacity, adding racks can make the system less efficient, because more compute creates more communication.

Its RoCE architecture for Vera Rubin is designed as a two-tier, non-blocking, multi-rail, multi-plane fabric. Each Rubin GPU is served by two ConnectX-9 SuperNICs, which the company says deliver up to 1.6 Tb/s of backend connectivity per GPU. According to CoreWeave, the modular design increases spine capacity within each plane and can extend the two-tier architecture to approximately 128,000 GPUs, avoiding extra network tiers that would add latency.

Scale-out testing deliberately turns off the fast GPU-to-GPU links and forces traffic over the backend network. CoreWeave then measures speed and consistency between every card, and checks average and worst-case latency under load. It also monitors for flaky connections, rising error rates, overheating hardware, uneven traffic and cabling that does not match the wiring map.

Stress tests that reflect real workloads

The final phase of bring-up runs workloads that stress the whole system at once. These include synchronized GPU communication, training-style jobs, heavy computation, custom benchmarks and traffic routed over the network fabric. CoreWeave also tests patterns that reflect agentic applications, such as sudden demand spikes, changing concurrency and bursts of communication.

The company says failures often emerge only when compute, networking, power, cooling and GPU-to-GPU communication are stressed together. Its stated benefit for customers is that by the time a rack reaches production, it has already handled the stress a customer workload will apply, so problems are caught in testing rather than during a live job.

What it means for customers

The post is a description of CoreWeave's own process, and the claims about performance and readiness are the company's. For customers running training, post-training or inference on Vera Rubin NVL72 capacity, the stated aim is faster time to a stable cluster and fewer surprises from stragglers, network issues or cooling faults in production.