CoreWeave's CAIOS hits 7+ GB/s per GPU on NVIDIA Blackwell Ultra

What CoreWeave tested

In March, CoreWeave published benchmark results for CAIOS on NVIDIA H100 GPU nodes, showing sustained throughput of over 2 GB/s per GPU across any number of GPUs. The company describes CAIOS as an AI-focused storage service designed to deliver higher throughput per GPU than traditional object storage and to scale to hundreds of thousands of GPUs. It includes the Local Object Transport Accelerator (LOTA), which prestages and caches objects on GPU nodes.

The new tests ran on 16 Blackwell Ultra nodes using the Warp S3 benchmarking tool inside CoreWeave Kubernetes Service. The setup used 10,000 objects of 50 MB each, with 15 MB parts, and a concurrency of 100. Pipelining was enabled, meaning cache reads were queued ahead across nodes instead of being made as sequential calls. CoreWeave tested both Ethernet (TCP) and RDMA over NVIDIA Quantum InfiniBand.

Ethernet versus InfiniBand results

On Ethernet, sustained throughput was capped at 180 GB/s across the 16 nodes, or 11.25 GB/s per node and 2.81 GB/s per GPU. CoreWeave said the Ethernet network in its test lab limited the result, so it could not reach the same per-node throughput as in the earlier test.

With RDMA over NVIDIA Quantum InfiniBand, sustained throughput reached 449 GB/s across the 16 nodes, or 28.06 GB/s per node and 7.02 GB/s per GPU. According to CoreWeave, the InfiniBand fabric easily handled full-fleet concurrency, which it presents as the scale advantage of RDMA transport. Throughput rose over time as objects were staged in the LOTA cache. Per-node maximums ranged from 25.6 to 36.9 GB/s. A GB300 node has 4 GPUs, so the 28.06 GB/s average works out to 7+ GB/s per GPU.

Where the gains came from

CoreWeave breaks the progression from 2 GB/s per GPU into three steps:

• Fewer GPUs per node: H100 nodes have 8 GPUs and Blackwell Ultra nodes have 4. With per-node throughput the same or higher, this doubles throughput per GPU, from 2 to 4 GB/s.
• InfiniBand for node-to-node transfer: Because the LOTA cache is global across all nodes, faster node-to-node transfer raises overall CAIOS throughput. CoreWeave reports an approximately 50% improvement, taking the figure from 4 to 6 GB/s per GPU. It says separate H100 InfiniBand testing showed similar gains over the March Ethernet results.
• LOTA pipeline optimizations: Changes to how LOTA reads and writes data gave an approximately 17% improvement, lifting throughput from 6 to 7+ GB/s per GPU. CoreWeave says recent LOTA tests on H100 nodes verified this.

These figures are CoreWeave's own, from its own test environment.

What it means for AI workloads

CoreWeave says CAIOS is built for the most data-intensive stages of AI: feeding large training sets into GPUs, shortening checkpointing and restore cycles, speeding up model weight loading, and supporting high-throughput key-value caches for inference. According to the company, the new throughput further reduces training times, lowers inference Time to First Token (TTFT) and improves ETL efficiency.

The company argues that the gap between GPU compute power and data availability will keep shrinking as it continues to optimize CAIOS, so that GPUs stay fully utilized at the extreme concurrency seen in training and inference. For customers, the results suggest the best throughput depends on the combination of Blackwell Ultra hardware, InfiniBand transport and pipelining, as the Ethernet run in this lab reached 2.81 GB/s per GPU.