vLLM adopts LibTorch Stable ABI to ease PyTorch upgrades
vLLM has migrated its CUDA build to PyTorch’s LibTorch Stable ABI, allowing its v0.33.0 LibTorch ABI Stable CUDA wheels to work across compatible PyTorch releases without rebuilding vLLM. The project says the change should let users upgrade sooner while reducing work for package distributors.
- vLLM publishes LibTorch ABI Stable CUDA wheels as of v0.33.0.
- Compatible PyTorch updates can be used without rebuilding vLLM.
- The migration moved some meta functions and AutogradFunctions to Python.
- The stable ABI does not remove Python, GPU or third-party compatibility constraints.
A change to how vLLM links with PyTorch
vLLM relies heavily on PyTorch and frequently calls LibTorch from its C++ backend. Previously, changes to LibTorch’s binary interface could require vLLM to be rebuilt and relinked, even when the source-level APIs remained unchanged.
The LibTorch Stable ABI provides a supported set of binary interfaces across compatible versions. vLLM says this means users can swap out the torch wheel and import it directly into an existing vLLM package. As of v0.33.0, vLLM publishes LibTorch ABI Stable CUDA wheels.
The stable interface has three layers: a C shim exposing selected LibTorch APIs, C++ wrappers that make the shim easier to use, and header-only utilities in torch::headeronly that do not link against libtorch.so.
The migration required changes across the CUDA build
vLLM began by inventorying its PyTorch API calls, identifying gaps in the stable APIs and planning replacements. Because the CUDA build contains over 175 .cpp and .cu source files, along with another 120+ headers, the project created a separate stable library and migrated kernels in manageable chunks.
The stable ABI does not currently support stable SymInt or expose PyTorch’s autograd machinery. As a result, vLLM moved its remaining meta functions and AutogradFunctions to Python. Operators also need to be registered through STABLE_TORCH_LIBRARY, with implementations wrapped using TORCH_BOX.
Many source changes were mechanical, such as replacing torch::Tensor with torch::stable::Tensor and using stable versions of existing operations. vLLM also created skills for AI agents that map existing APIs to stable equivalents and flag common migration mistakes. The project says it has not yet developed evaluations to measure those skills’ effectiveness.
Compatibility still has limits
The stable ABI covers only a subset of LibTorch. When a stable equivalent for is_pinned() was unavailable during the migration, vLLM used the general-purpose torch_call_dispatcher C shim to call the ATen operation through the dispatcher. The source says PyTorch added a stable C++ is_pinned() method in version 2.14.
The shim can add host-side overhead through type conversion, operator lookup and boxing. According to vLLM, this is usually negligible and disappears after warmup under CUDA Graphs, although it can matter in eager mode when repeatedly launching very fast kernels. The project says it saw no noticeable runtime regression during the migration.
vLLM stresses that binary compatibility is only one part of compatibility between vLLM and PyTorch. Python APIs, GPU requirements such as the minimum supported CUDA version, and third-party dependencies can still impose constraints. The project uses torch-abi-audit in continuous integration to inspect compiled libraries, identify unstable PyTorch symbols and report the minimum LibTorch version required by stable symbols.
vLLM: Migrating vLLM to the LibTorch Stable ABI
