CoreWeave Reports 6.5x DeepSeek R1 Inference Gain on NVIDIA GB300 NVL72

How CoreWeave ran the benchmark

CoreWeave tested the DeepSeek R1 reasoning model in a head-to-head comparison. One side was a 16-GPU NVIDIA H100 system. The other used only four GPUs on the new NVIDIA GB300 NVL72 infrastructure.

According to CoreWeave, the memory and interconnect bandwidth of the GB300 let the model run with 4-way Tensor Parallelism (TP4). The H100 setup needed 16-way Tensor Parallelism (TP16). The company says this cut communication overhead sharply and produced over 6x higher raw throughput per GPU on the GB300. The headline figure it reports for inference on DeepSeek R1 is a 6.5x improvement.

These are results published by CoreWeave itself, and the post does not describe an independent verification.

Why reasoning models stress inference

CoreWeave frames the test around the industry shift from simple generative models to reasoning models. Models like DeepSeek R1 can perform complex, multi-step tasks, analyze data and act as autonomous agents. That capability, the company argues, comes at a steep computational cost during inference.

The chain-of-thought processes behind this behavior need many iterative steps, which makes them very sensitive to latency. CoreWeave says a slow, high-latency experience can make an AI agent feel unusable, and that lower communication overhead lets these processes run in a fraction of the time. In its view, this makes real-time AI agents and sophisticated reasoning applications viable.

What is inside the GB300 NVL72

The GB300 NVL72 is a rack-scale system designed for frontier AI models. CoreWeave highlights three elements:

• Memory: 37TB of total memory, which the company says lets customers handle larger and more complex models.
• Interconnect: fifth-generation NVIDIA NVLink delivering 130TB/s of bandwidth to the 72 interconnected NVIDIA Blackwell Ultra GPUs.
• Networking: NVIDIA Quantum-X800 InfiniBand as the scale-out compute fabric across the cluster.

The software layer on CoreWeave's cloud

CoreWeave argues that hardware alone does not deliver the result. It describes several parts of its platform that support the instances:

• A proprietary Rack LifeCycle Controller that automates health verification, firmware provisioning and system imaging, covering components from compute trays to NVLink switches.
• Integration with Kubernetes (CKS) and Slurm on Kubernetes (SUNK), plus a topology-aware scheduler. Using NVLink labels and a custom block plugin, the control plane schedules workloads to run within the same NVL72 domain whenever possible, so jobs are not scattered across racks.
• Managed Grafana dashboards for real-time monitoring of GPU utilization, NVLink traffic and rack availability.

What it means for customers

CoreWeave says the gains translate into faster training of larger models, more inference throughput and responsiveness for the same budget, and a platform built for reliability at frontier scale. The post points to reduced total cost of ownership, though it gives no pricing or cost figures.

For enterprises weighing infrastructure for reasoning workloads, the main takeaway is the claimed reduction in parallelism needed: four GB300 GPUs doing the work of a 16-GPU H100 setup in this test. The company directs interested customers to contact it about the GB300 NVL72 instances. Availability terms and pricing are not included in the announcement.