Artificial intelligence news.
Straight from the original announcements.

PyTorch team integrates SGLang with IBM Spyre in less than a week

November 12, 2026, 15:45By the AI News Bites Editorial TeamSource: PyTorch
PyTorch team integrates SGLang with IBM Spyre in less than a week
Image: PyTorch

A two-person team integrated SGLang with IBM Spyre in less than a week, using AI coding agents to build an open source plugin of roughly 1,500 lines. PyTorch says SGLang delivered about 3.6× the throughput of a reference PyTorch inference implementation using torch-spyre.

Key points
  • A two-person team integrated SGLang with IBM Spyre in less than a week.
  • The open source plugin contains roughly 1,500 lines of code.
  • SGLang delivered approximately 3.6× the throughput of a reference PyTorch implementation using torch-spyre.
  • SGLang was on par with vLLM on measured Granite 3.3 8B and Gemma 4 26B A4B workloads.
  • RadixCache prefix sharing worked without Spyre-specific changes.

A short integration built on existing software

The team wanted to make SGLang’s serving stack, including its built-in prefix sharing, available to users of IBM’s Spyre accelerator for enterprise AI inference. Spyre integrates with PyTorch through torch-spyre, which registers it as a PyTorch device and connects it to torch.compile.

Existing PyTorch hardware support and SGLang’s extension points limited the amount of new code required. AI coding agents used existing PyTorch implementations of Spyre operations as reference material, then adapted the integration code to SGLang’s backend interfaces. Their role focused on connecting established hardware support to the serving framework.

The result was a plugin of roughly 1,500 lines. Four small patches extended SGLang’s backend support. Two exposed existing execution paths without CUDA-specific dependencies, while two registered Spyre as a supported device and attention backend that does not use Triton. The compatibility changes could also support other accelerators, while the registration changes are specific to Spyre.

Performance compared with PyTorch and vLLM

According to PyTorch, SGLang on Spyre achieved approximately 3.6× the throughput of the reference PyTorch inference implementation using torch-spyre. The team also compared SGLang with vLLM on Spyre using Granite 3.3 8B and Gemma 4 26B A4B.

Across the workloads measured for those two models, SGLang’s performance was on par with vLLM. The post says the comparison used relative performance normalized to vLLM = 1.0, with lower values representing better performance.

The integration reused attention code from the existing vLLM Spyre backend, including its handling of input buckets and tensor shapes. It also reused the backend’s PyTorch-based approach to allocating the KV cache. Existing Spyre implementations for RMSNorm, SiLU, GeLU, rotary embeddings and linear layers were connected to SGLang’s operator dispatch.

What the approach means for Spyre users

Some SGLang features required no Spyre-specific changes. RadixCache, which provides prefix sharing, is handled by SGLang’s scheduler before execution reaches the accelerator, so the existing scheduling logic worked unchanged. SGLang’s device-agnostic KV pool likewise required only a single method override for buffer creation, although allocation still needed PyTorch-based handling where the existing path relied on Triton.

The plugin is available as open source alongside upstream SGLang. PyTorch says future Spyre software changes are expected to add dynamic shapes over the coming months, reducing some of the shape management required today. Later this year, the team also expects Triton support for Spyre and RadixAttention on top of it.

For users, the work provides a route to run SGLang’s serving stack on Spyre while retaining shared PyTorch support and features such as prefix sharing. For developers, the post presents the integration as a repeatable model: keep hardware support in PyTorch and connect serving frameworks through defined extension points.

Related news

PyTorch and TorchTitan Train a 1T-Parameter MoE on JUPITER
ResearchNovember 12, 2026, 13:00

PyTorch and TorchTitan Train a 1T-Parameter MoE on JUPITER

The EUROPA consortium says it has scaled pretraining for a roughly 1-trillion-parameter Mixture-of-Experts model on JUPITER, a publicly funded European supercomputer.…

PyTorch serves Meta’s 816B Muse Spark Open on one TPU v7 host
For developersNovember 10, 2026, 17:45

PyTorch serves Meta’s 816B Muse Spark Open on one TPU v7 host

PyTorch says it can serve Meta’s 816B-parameter Muse Spark Open mixture-of-experts model on a single TPU v7 host containing four chips. The system uses SGLang-TorchTPU…

vLLM adopts LibTorch Stable ABI to ease PyTorch upgrades
For developersNovember 6, 2026

vLLM adopts LibTorch Stable ABI to ease PyTorch upgrades

vLLM has migrated its CUDA build to PyTorch’s LibTorch Stable ABI, allowing its v0.33.0 LibTorch ABI Stable CUDA wheels to work across compatible PyTorch releases…