Ray Launches Special Interest Group to Shape Accelerator Support

Why Ray is setting up an accelerator group

According to the announcement, the computing landscape was very different when Ray was created at UC Berkeley more than 10 years ago. At that time, "distributed computing" meant CPU clusters running frameworks such as Apache Hadoop and Apache Spark, and GPUs were only beginning to move from graphics and scientific computing into machine learning.

Today accelerated computing is central to AI development and deployment, and the hardware has become more specialized and diverse. The post lists several examples:

• NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs, which support workloads spanning AI training, inference, graphics and video.
• NVIDIA HGX B200, which offers substantially greater memory capacity for larger models, and NVIDIA GB200 NVL72, which connects 72 NVIDIA Blackwell GPUs for communication-intensive, large-scale training and inference.
• AMD MI355X, which also provides substantial memory capacity for large models.
• Google Ironwood TPUs, which target massive-scale AI workloads.
• NVIDIA Groq 3 LPUs, which specialize in low-latency inference.

Ray already supports more than a dozen kinds of hardware accelerators, from NVIDIA and AMD GPUs to TPUs, Trainium, Gaudi and others. The new SIG is meant to build on that base with a more systematic effort to make accelerators easy to use, efficient at scale and reliable in production. It will bring together hardware vendors, Ray contributors and application developers.

Three priority areas: testing, scheduling and networking

The initial roadmap covers three areas. The first is development and testing infrastructure. The SIG says reliable accelerator support needs dependable hardware access so contributors can validate changes, reproduce failures and catch regressions. It adds that as agentic software engineering speeds up development, both human contributors and coding agents need reproducible feedback from real hardware. Planned work includes shared hardware access through a documented process for bringing cloud and vendor-hosted accelerators into Ray's infrastructure, maintained environments with explicit driver, runtime and dependency versions, broader CI coverage that includes distributed workloads, failure recovery and performance regressions, and clear ownership for maintaining the setup.

The second area is hardware-aware scheduling. The post argues that accelerator nodes are no longer just a count of identical devices, because performance depends on how devices are interconnected, how they relate to CPUs and NICs, and how nodes are grouped into racks and TPU slices. The goal is to give Ray an accurate, consistent picture of this structure across vendors and platforms, so the scheduler can place tasks and actors on suitable hardware. Priorities include common vendor-neutral interfaces for labeling and describing hardware, a documented extension point and checklist for vendors adding a new accelerator, hardened discovery that captures topology and architecture, cooperation with cloud providers, since some information such as rack or slice membership is only available from the platform, hardware views alongside accelerator metrics on the Ray dashboard, and clearer user guides.

Faster data movement between devices

The third area is networking and I/O transport. The announcement notes that moving data between accelerators is often as important as the computation itself. Workloads such as large scale reinforcement learning must move large model weight tensors from training to inference workers across devices and nodes.

Ray's object store was historically built for CPU data, so accelerator tensors had to be copied to host memory, serialized and sent over standard host networking. Ray Direct Transport (RDT) was introduced to let actors pass references to tensors through Ray while the data itself moves directly between devices, using high performance transports such as UCX and RDMA.

The SIG wants this to work reliably across different accelerators, frameworks and network fabrics while keeping Ray's familiar programming model. Priorities include supporting more accelerators, frameworks and interconnects in RDT so Ray can pick the fastest path, such as intra-slice TPU ICI, scale-up NVIDIA NVLink or scale-out GPUDirect RDMA. They also include clean initialization, teardown and error recovery for each transport, and validated deployment recipes with benchmarks, diagnostics and transport metrics published by cloud and hardware providers.

Scope and how to take part

Core scheduling and placement logic stays in Ray Core. SIG Accelerator will be responsible for defining the plug-in interfaces, CI testing and maturity tiers for accelerators in Ray. Hardware vendors and cloud providers are expected to contribute and maintain their own device plugins, transports and day-zero support for new SKUs, while framework authors and ML practitioners keep the roadmap grounded in real workload needs.

The group invites accelerator integrators, teams running large-scale training or inference on Ray, and providers of cloud and CI infrastructure to join. Participation options include the Ray Slack, community meetings to discuss proposals and review roadmap priorities, and open accelerator-related issues and pull requests on GitHub.

For users, the stated aim is a clearer, vendor-neutral path for new hardware to be supported in Ray, with better testing and visibility into how clusters are built. The post does not give timelines for any of the roadmap items.