Ray Gets a Special Interest Group to Shape Vendor-Neutral Accelerator Support

Why a dedicated group for accelerators

The announcement, published on the Anyscale blog on October 20, 2026 by Zhe Zhang, Mengjin Yan, Ryan O'Leary (Google) and Richard Liaw, looks back at how much has changed since Ray was created at UC Berkeley more than 10 years ago. Back then, "distributed computing" meant CPU clusters running frameworks such as Apache Hadoop and Apache Spark, and GPUs were only beginning to move into machine learning.

Today, the authors write, accelerators have become more specialized and diverse. They point to several examples: NVIDIA's RTX 6000 Pro for flexible workloads spanning AI training, inference, graphics and video; NVIDIA B200s and AMD MI355X with substantially greater memory capacity for larger models; and GB200 NVL72, which connects 72 GPUs for communication-intensive training and inference. Beyond GPUs, Google Ironwood TPUs target massive-scale AI workloads, while Groq LPUs specialize in low-latency inference.

According to the post, Ray already supports more than a dozen kinds of hardware accelerators, from NVIDIA and AMD GPUs to TPUs, Trainium, Gaudi and others. The new group is meant to build on that base with a more systematic effort to make accelerators easy to use, efficient at scale and reliable in production.

Three areas of the initial roadmap

The first focus is development and testing infrastructure. The group wants a documented process for bringing cloud and vendor-hosted accelerators into Ray's testing setup, maintained environments with explicit driver, runtime and dependency versions, and broader CI coverage. That coverage should include distributed workloads, failure recovery and performance regressions, not only basic device functionality. The post notes that as agentic software engineering speeds up development, both human contributors and coding agents need reproducible feedback from real hardware.

The second area is hardware-aware scheduling. The authors say accelerator nodes are no longer just a count of identical devices, since performance depends on how devices interconnect, how they relate to CPUs and NICs, and how nodes group into racks and TPU slices. Planned work includes vendor-neutral interfaces for labeling and describing hardware, including a consistent way to identify accelerator SKUs at launch, stronger discovery that captures topology and architecture, and cooperation with cloud platforms, since some information, such as which rack or slice a node belongs to, is only available from the platform. The group also plans views of accelerators and their connections alongside metrics on the Ray dashboard, plus clearer user guides.

The third area is networking and I/O transport. Ray's object store was originally built for CPU data, so accelerator tensors had to be copied to host memory, serialized and sent over standard networking. Ray Direct Transport (RDT) was introduced to let actors pass tensor references through Ray while the data moves directly between devices, using transports such as UCX and RDMA. The group wants RDT to support more accelerators, frameworks and interconnects, including intra-slice TPU ICI, same-node NVLink and cross-node GPUDirect RDMA, and to handle initialization, teardown and error recovery reliably in workloads like reinforcement learning and disaggregated serving.

Scope and how to take part

The post draws a line between the group and Ray Core. Core scheduling and placement logic stay in Ray Core, while SIG Accelerator is responsible for plug-in interfaces, CI testing and maturity tiers for accelerators. Hardware vendors and cloud providers are expected to contribute and maintain their own device plugins, transports and day-zero support for new SKUs. Framework authors and ML practitioners are meant to keep the roadmap grounded in real workload needs.

For users, the stated aim is that applications and libraries can target hardware capabilities rather than write vendor-specific code, and that new accelerators work with Ray at launch. The source gives no timelines for these goals.

Anyone building an accelerator integration, running large-scale training or inference on Ray, or providing cloud and CI infrastructure is invited to join. The authors point to the Ray Slack, community meetings for discussing proposals and roadmap priorities, and open accelerator-related issues and pull requests on GitHub.