PyTorch Accelerator Group Reports H1 2026 Progress on Testing and CI

A relay to close the CI visibility gap

The Cross-Repository CI Relay addresses a coordination problem in PyTorch's continuous integration. PyTorch sits at the center of many dependent projects, including accelerator backends such as Intel XPU, AMD ROCm, Apple MPS and Qualcomm AI Engine, as well as libraries such as vLLM, SGLang and Hugging Face Transformers. The upstream CI runs inside the pytorch/pytorch repository, so downstream repositories had no standard way to know when to test against upstream changes or to report results back.

As a result, PyTorch contributors could not tell whether their pull requests would affect other backends before merging, and hardware maintainers could only give feedback after changes were committed, the post says.

With CRCR, opening a PR or pushing a commit to pytorch/pytorch triggers a webhook that dispatches events to all registered downstream repositories in parallel. Each repository runs its own CI workflow and reports back through an authenticated callback that uses a GitHub OIDC token to verify the caller's identity. Results appear in the PyTorch CI HUD within seconds, giving maintainers one dashboard for in-tree and cross-repository CI health.

Participation is tiered across four levels, L1 to L4. Downstream repositories can move from simple dispatch notifications to full HUD reporting, then to non-blocking and eventually blocking check runs on upstream PRs. The group describes five layers of validation, including OIDC identity verification, allowlist authorization and rate limiting, so that even a compromised downstream repository can only affect its own displayed CI status. Onboarding needs only an allowlist entry and a lightweight workflow file, with no custom authentication code.

Making 600,000 tests hardware-agnostic

PyTorch maintains over 600,000 test cases covering operators, autograd, profiling and distributed training. Many were written for a specific set of accelerators, with device strings, profiler activities, memory APIs and skip decorators hardcoded. Before the refactor, backend vendors had to maintain custom test cases via patches, which demanded significant manual effort at every PyTorch upgrade, a burden made heavier by the faster release cadence.

In H1 2026 the group began decoupling the suite from specific hardware. The work is tracked in a central issue, two RFCs and a project with 259 items. It has three parts:

• Device-agnostic migration: hardcoded references such as device="cuda" and @onlyCUDA are replaced with parameterized equivalents, including instantiate_device_type_tests(). Test classes are labeled accelerator-unrelated, accelerator-agnostic or accelerator-specific. Contributors migrated over 276 test files spanning areas such as dynamo, profiler, nn modules, linalg and optimizers.
• Hardware classification metadata: a hw_classification class attribute (GENERIC, DEVICE_GENERIC, CUDA, XPU, MPS) and a --hw-classification flag let CI runners select the right test subset for their hardware automatically.
• CI guardrails: a HW_CLASSIFICATION linter requires every new test class to declare its classification. The 1,191 unclassified files present at launch are allowlisted and are being reduced toward zero, with a future AI linter layer planned.

What it means for accelerator vendors and next steps

For accelerator developers, the group says the changes shift onboarding: instead of patching many test files, vendors can rely on instantiate_device_type_tests() to generate test variants for their backend. Tests previously invisible to non-CUDA backends now run on any registered accelerator, and a single device-agnostic test can produce validated configurations across CUDA, MPS, XPU, ROCm and PrivateUse1 backends at once. A multi-level skipping mechanism, by feature, class, test and operator, lets vendors choose which tests to run based on their implementation maturity without changing upstream test code.

For the PyTorch project, the classification system is the foundation for class-level hardware and frequency metadata intended to reduce double-testing and improve CI observability. The H1 migration is described as a strict prerequisite for that vision.

In H2, priorities include driving the unclassified allowlist toward zero, extending device-agnostic coverage to distributed, JIT and autograd modules, and wiring hw_classification into CI scheduling. The group is also exploring a declarative per-operator capability registry so accelerators can declare supported dtypes and precision overrides without editing upstream OpInfo entries.

The post also lists work on profiling support for PrivateUse1-based backends, the OpenReg reference backend, distributed capability support and compiler backend integration. The available text covers those areas only in brief.