CoreWeave adds torchforge support to scale reinforcement learning on its cloud

What torchforge does

According to CoreWeave, torchforge simplifies reinforcement learning (RL) by separating algorithm design from distributed infrastructure. The company says this lets researchers scale complex RL workloads to thousands of GPUs with minimal code.

RL workflows combine continuous inference and training in a tightly coupled loop: the model generates responses to prompts, a reward model scores them, and the base model is updated with those scores. CoreWeave describes this as difficult to orchestrate at scale. Researchers have to manage separate inference and training stacks, shard models across GPUs and nodes, handle failures and transfer weights efficiently between the two phases.

torchforge implements GRPO, a popular RL algorithm aimed at improving a model's reasoning quality. It is built on Monarch, a PyTorch-native distributed programming framework, and uses RDMA for fast, fault-tolerant data movement. CoreWeave says researchers can write RL code almost as simply as pseudocode while the framework handles scaling, routing, load balancing and fault tolerance.

A 512-GPU test with Meta and Stanford

Meta, Stanford's Scaling Intelligence Lab and CoreWeave ran a large-scale post-training job of a state-of-the-art dense coding model on a cluster with 512 NVIDIA H100 GPUs. CoreWeave says the run confirmed torchforge's stability, performance and production-grade functionality at scale.

In the collaboration with Stanford's lab, dozens of torchforge runs were executed on CoreWeave's infrastructure, including full-scale experiments that used all 512 H100 GPUs.

CoreWeave frames RL as the leading post-training technique for improving model performance while reducing serving costs. Unlike supervised fine-tuning, which teaches a model to imitate patterns in labeled data, RL trains from outcomes using feedback and rewards. The company says this allows smaller models to match larger ones on specific tasks while being cheaper and faster to run.

Why SUNK is the scheduler

CoreWeave argues that shared research clusters are hard to keep well utilized: multi-node jobs must launch together, inference and training load the network fabric differently, long generations and verifiers can slow tasks, and frequent weight syncs can hurt performance if the network topology is not optimized. Slurm addresses these problems with priorities, preemption, quotas, gang scheduling and topology-aware scheduling. But managing Slurm itself becomes complex when node counts reach the thousands.

SUNK combines Slurm scheduling with CoreWeave's managed Kubernetes service, CKS. According to the company, it supports jobs spanning more than 32,000 GPUs, keeps Slurm components highly available, scales compute nodes on demand and replaces the Slurm controller API to handle hundreds of thousands of jobs.

Workflow stays familiar for Slurm users. A torchforge job is defined in a standard Slurm batch script and launched with sbatch or srun. SUNK handles GPU allocation and node scaling, while Kubernetes restarts failed components. If a node fails or a service restarts, the job continues thanks to Monarch's fault-tolerant controller and torchforge's built-in retry logic, CoreWeave says.

SUNK also offers Individual Login Nodes, in-cluster file systems, tooling for building container images and IdP-federated access management through Automated User Provisioning.

The CoreWeave stack and what comes next

CoreWeave points to several infrastructure components behind the results: NVIDIA Quantum InfiniBand networking, CKS running on bare metal, NVIDIA DPUs that offload networking and storage traffic, and Mission Control for cluster validation, health monitoring and rapid node replacement. Its AI Object Storage with Local Object Transfer Accelerator (LOTA) sustains data loading and checkpointing at about 7 GB per GPU per second across thousands of GPUs, according to the company. Observability comes through Managed Grafana Dashboards and an integration with Weights and Biases.

CoreWeave claims the combined stack delivers up to 10x better reliability, measured by mean-time-to-failure, 20% higher Model FLOPs Utilization and 96% Goodput, which it defines as effective training time ratio.

torchforge is now supported on CoreWeave through SUNK, and the company has published instructions for installing it in a SUNK cluster and running a simple RL job. CoreWeave describes the work with Meta and Stanford as the beginning of a broader partnership, and says it plans to work with Meta's PyTorch team on performance, reliability and developer experience for RL workloads, and to extend the collaboration to other parts of the PyTorch ecosystem.