PyTorch Models Can Now Run on GPUs Inside Web Browsers
PyTorch models can now run on a device’s GPU inside the web browser through the new ExecuTorch WebGPU backend. The approach uses a single ExecuTorch .pte file, avoids conversion to formats such as ONNX or GGUF, and keeps model inputs and outputs on the device.
- ExecuTorch WebGPU runs PyTorch models on browser GPUs.
- Models are exported to a single ExecuTorch .pte file.
- Inference runs locally, so data does not leave the device.
- WebGPU was 4, 7× faster on prefill and 2, 10× faster on decode in PyTorch’s benchmark.
- The backend supports language, vision, speech and on-device training models.
A direct path from PyTorch to the browser
Developers export models with torch.export() and ExecuTorch’s WebGPUPartitioner, producing an ExecuTorch .pte file. That same file can run in WebGPU-enabled browsers through the ExecuTorch WebAssembly runtime, including browsers such as Chrome and Safari.
The backend uses WGSL compute shaders and WebGPU’s support for compute shaders, shared memory, storage buffers and barriers. It translates WGSL for different platforms, including Metal on macOS, Vulkan on Linux and Android, and D3D12 on Windows.
For browser deployment, the C++ runtime is compiled to WebAssembly with Emscripten. The model can then be served over a content delivery network on a web page, without requiring users to install a native application or connect to a backend server.
Local inference across a broad set of models
According to PyTorch, ExecuTorch WebGPU supports a wide range of model types. Examples include Llama 3.2, Qwen3 and Gemma 3 for large language model inference, SmolVLM for image question answering and captioning, and Whisper for speech-to-text.
The validation list also includes Vision Depth Anything V2 for monocular depth estimation, SAM 2.1 Hiera Tiny for promptable image segmentation, D-FINE for object detection, Real-ESRGAN for super-resolution and MODNet for foreground segmentation. Phonon-31M-EN supports text-to-speech.
The source also describes on-device LoRA and DiReFT fine-tuning for Llama 3.2 1B, Qwen2.5 0.5B and Qwen3 0.6B. These capabilities are aimed at personalized assistants and domain adaptation on-device.
Performance and privacy benefits
PyTorch benchmarked WebGPU against its own WebAssembly CPU backend, which uses multi-threaded XNNPACK, on the same machine. Using quantized Llama 3.2 1B and Qwen3-0.6B in Google Chrome on an Apple M1 Pro, WebGPU was 4, 7× faster on prefill and 2, 10× faster on decode, according to the company.
The measurements used the median of 5 runs after 3 warmups, across 128 to 8K tokens of context. PyTorch says the models use identical token IDs for both backends, while the two backends use different activation configurations.
Running inference in the browser can reduce network latency and support offline use cases. It also keeps prompts, photos, camera frames, audio and other model data on the user’s device, which can reduce server-side inference costs and improve privacy.
The backend is still in early development
Users can try the ExecuTorch WebGPU Playground, which runs many of the listed models on the same WebAssembly and WebGPU runtime. Each model’s Hugging Face repository includes its .pte files, a model card and the exact export recipe.
PyTorch says the backend is still in early development. The project plans to improve its speed and stability and expand operator, data type and model coverage. ExecuTorch is open source, and the project is inviting contributions, bug reports and feedback through its issue tracker and Discord community.
PyTorch: ExecuTorch WebGPU: PyTorch Models on the GPU, in the Browser
