Anyscale and CoreWeave caption 600 TB of video in 95 minutes on 1,600 GPUs

Two layers, one pipeline

CoreWeave argues that data processing for multimodal AI has become a GPU workload. Video, audio, LiDAR and satellite data must be decoded, filtered, featurized and normalized, and at exabyte scale the company says that requires GPUs. Standing up such infrastructure usually costs a team weeks or months, according to the post.

The test split the work into two layers. Ray OSS and Anyscale handle the workload: scheduling tasks and actors, streaming data through the pipeline and scaling workers to demand. Ray cannot see hardware problems such as a throttling node or a straggler GPU. That is where CoreWeave comes in.

CoreWeave Kubernetes Service runs Kubernetes on bare metal nodes with GPU drivers, network and storage interfaces and observability plug-ins already installed. CoreWeave says this is why the Anyscale install took an hour instead of a quarter. CoreWeave Mission Control monitors node and cluster health and replaces unhealthy nodes automatically. CoreWeave AI Object Storage (CAIOS) holds the data in a single global namespace with no egress, request or transaction fees, and its LOTA component caches fetched objects on local NVMe.

From signup to production in under 24 hours

CoreWeave laid out the timeline hour by hour:

• Hour 0: a reservation was provisioned, scaling to 1,600 GPUs, with 200 AMD Turin 9655P nodes at 192 vCPUs each and CAIOS as the data lake.
• Hour 1: Anyscale was installed on CKS as an operator.
• Hour 2: an orchestration test covered node management and Ray cluster lifecycle.
• Hour 10: the pipeline was written in Ray Data as a single streaming job.
• Hour 15: tuning for GPU utilization and throughput.
• Hour 20: the same code was promoted to a fault-tolerant Anyscale Job with no rewrite.

The pipeline had a CPU stage for video decode, scene-boundary splitting and filtering, and a GPU stage that ran caption and annotation inference with Qwen3-VL-8B.

The first version used Ray Core with hand-managed actors, each holding one GPU and one vLLM engine, on a fixed list. Moving to Ray Data replaced that with streaming execution, autoscaling and built-in batched inference. On a 600 GB dataset, processing 43,700 clips into 1 million captions, Ray Data used 73% of the GPU time Ray Core needed and produced 3.7x more captions per GPU-hour, according to the post.

Scaling the data 1,000x

To test cost at scale, the team grew the 600 GB dataset to 600 TB by generating 1,000 synthetic variations of each video with FFmpeg. At peak, 35,820 CPU cores turned 43,700 original clips into 70 million. CAIOS absorbed the writes at 40 GB/s, saturating network bandwidth, with PUT latency at a p50 of 9.1s and a p99 of 18.8s.

For captioning on the full 600 TB, CAIOS served 110 to 120 GB/s across roughly 46 CPU nodes without caching, and the job finished in 95 minutes. Scaling the GPU cluster roughly 6x, from 256 to 1,600 GPUs, kept captions per GPU-hour steady. Total throughput grew 10.2x, from 1,273 to 12,666 captions per second. CoreWeave says the larger CPU pool kept decoding ahead of the GPUs.

Loading models without a storage bottleneck

Loading Qwen3-VL onto every GPU at once moves about 4.3 TB of tensor data, with up to 255 peers requesting the same bytes simultaneously. CoreWeave says this pattern flattens conventional object storage. Because LOTA caches on node-local NVMe, later engines asking for the same tensor do not touch the backend.

For the 17 GB Qwen3-VL model, each engine read at 8.68 GB/s and the full model landed in about two seconds. CoreWeave puts that at 1.88x faster than third-party object storage and 1.28x faster than a cold read from CAIOS. For raw ingestion, 1,357 parallel tasks read 677 GB of MP4 files at about 4.5 GB/s, finishing in 140 seconds.

What it means for teams

The results are the companies' own benchmarks, but they point to the claim at the center of the post: that infrastructure setup, not pipeline code, is often the slow part of preparing multimodal training data. For teams building video datasets for areas such as physical AI, the reported figures suggest a managed GPU cloud paired with Ray Data can cut setup from weeks to a day and keep GPUs busy as data grows by orders of magnitude.