CoreWeave benchmark: AI Object Storage tops 2 GiB/s of read speed per GPU

What CoreWeave tested

CAIOS is an exabyte-scale, S3-compatible managed cloud storage service built for GPU-intensive AI model training. CoreWeave announced its general availability in March. A central part of the service is the Local Object Transport Accelerator (LOTA), which caches frequently accessed or pre-staged data on local NVMe disks inside GPU nodes. According to CoreWeave, this reduces network latency and speeds up training workflows.

For the benchmark, the company deployed a test application on 20 CoreWeave GPU nodes in its US-WEST-04A Availability Zone. Each node had 8 NVIDIA H200 GPUs, one NVIDIA BlueField-3 DPU with dual 100Gbps redundant network links for storage access, and 8 NVIDIA ConnectX-7 InfiniBand adapters. Each node had a 1 TiB local cache, which gave a 20 TiB global cache across the cluster. All reads and writes went through the cwlota.com endpoint.

CoreWeave notes that the tests used CPU-driven I/O rather than direct GPU storage reads. They ran on H200 nodes to match real-world production environments.

Method and results

The team used the open-source Warp S3 benchmarking tool, which simulates read and write scenarios with randomly generated data. Each Warp instance on each node wrote 10,000 objects of 50 MiB with 10 MiB parts, using 100 concurrent threads. Warp then ran random reads across the 20 nodes for 10 minutes. Because the same objects were accessed repeatedly from several nodes, the LOTA cache could accelerate later reads.

In the first minute, while the cache was still filling, CoreWeave observed an aggregate read throughput of about 24 GiB/s, or 1.2 GiB/s per node. Within another minute the cache was full and all later reads came from local NVMe storage. Throughput then rose to 368 GiB/s in total, an average of 18.4 GiB/s per GPU node or 2.3 GiB/s per GPU.

CoreWeave says this shows CAIOS can exceed 2 GiB/s of read performance per GPU, and that customers operating at thousands of GPU nodes corroborate the figure. The company argues that because cached or pre-staged data is read at the speed of the node-local storage, performance should scale to any number of GPU nodes and GPUs. That scaling is an expectation stated by CoreWeave, not something the 20-node test measured directly.

Caching behavior and pre-staging

According to CoreWeave, once any segment of an object is accessed, the whole object is cached for later reads. The company says this helps in workloads where partial reads are common, such as autonomous vehicle video training or checkpoint restoration.

Users can also load datasets into the cache ahead of time. CoreWeave names large multimedia files, inference applications and container registry images as cases where this is useful. Existing S3-compatible workloads only need to be pointed at the dedicated endpoint, and the company says no additional coding is required.

What it means for AI teams

The results are CoreWeave's own, from a test it designed and ran, and they cover reads only. The company says its next benchmarking phase will examine write performance, which matters for checkpoint operations, and that it expects write throughput to be similarly strong given the horizontally scalable design.

For data scientists and ML engineers, the practical claim is that data access should not hold back GPUs as models and datasets grow. CoreWeave also describes the test setup in enough detail, including the Warp parameters and the Helm-based deployment, for customers to try to reproduce it in their own environments.