PyTorch 2.15 brings FP8 and much faster MoE decoding to Macs

FP8 arrives on the Mac

The headline addition is support for torch.float8_e4m3fn tensors on MPS. Each value takes one byte, which is half the storage of FP16 or BF16. Such tensors can be multiplied with F.scaled_mm, which supports both tensorwise and rowwise scaling. The team says other operations that are supported on CUDA and CPU work with FP8 on MPS as well.

The post was written by the PyTorch team at Hugging Face, which contributes to PyTorch and focuses in part on the MPS backend. According to the authors, early 2026 meant working around unsupported operations, performance cliffs and unexpected memory growth even for small models on MPS. Their approach is to run real models from the Hugging Face Hub, find where MPS falls short and fix it upstream, increasingly by replacing MPSGraph with dedicated Metal kernels.

Faster decoding for language models

During the decode phase of an LLM, each new token passes through the model as a single row, so large matrix multiplications become matrix-vector products. PyTorch 2.15 speeds up the two layer types that dominate this step.

For dense layers, GEMV kernels have existed for torch.mm since 2.14, but F.linear still went through MPSGraph. Version 2.15 connects them, with eligible bias additions fused in. Across 96 BF16 layer shapes from real models, the team reports a 1.23× geometric-mean speedup. The biggest gains are on small projections such as MoE gates, while large layers and LM heads stay near 1× because they were already close to bandwidth-bound. Softmax over the vocabulary is about 8× faster.

For mixture-of-experts layers, 2.15 provides a native grouped matmul. In 2.14, group offsets were copied to the CPU and the groups were looped over. The new Metal kernels read the offsets directly on the GPU, and small groups, common in decoding, get their own kernel. On an M3 Ultra with torch.compile, the post reports that Qwen3.6-35B-A3B goes from 9.4 to 61.3 tokens/s and LFM2.5-8B-A1B from 23.6 to 123.6 tokens/s.

Linear algebra, audio, video and training

Complex64 LU factorization, LU solve and triangular solve now run on MPS, which enables operations built on them, such as complex determinants. Batches of FP32 matrices up to 4 × 4 are inverted in one Metal kernel, QR uses blocked updates, and Gated DeltaNet workloads get a dedicated path for 16 × 16, 32 × 32 and 64 × 64 FP32 triangular systems with at least eight right-hand sides. SVD now handles very small singular values and extreme input magnitudes more robustly.

Several operations used in audio, video and training get new Metal paths, including conv1d forward, constant padding, dropout, interpolation backward and reductions. The padding change has visible effects:

• Decoding a video with Wan2.2's tiled VAE used to push driver memory to 270 GiB. It now peaks at about 8 GiB.
• On 2.14, the Qwen-Image-2.1 VAE encoder silently corrupted images on MPS, as reported in ComfyUI. In 2.15, an encode-decode round trip matches CPU at 29.95 dB PSNR, up from 10.06 dB.

The release also adds public MPS streams and more reliable event timing from Python.

Requirements and what it means for Mac users

MPS now requires macOS 15 or later, and PyTorch 2.15 requires Python 3.11 or later. An Apple silicon Mac is needed. If the OS is too old or no Metal GPU is available, the MPS availability check now explains why. The release candidate can be installed in a separate environment with uv pip install torch==2.15.0 --index-url https://download.pytorch.org/whl/test/cpu.

For people running models locally with transformers, diffusers or tools like ComfyUI, the authors point to faster local LLMs, video generation that fits in memory, correct image editing with Qwen-Image-2.1 and FP8 weights at half the memory of BF16. Dense models benefit too, through faster linear layers and softmax.

Looking ahead, the team previews PyTorch 2.16 with attention fully on Metal, more MPSGraph-to-Metal migrations and faster LLM decoding.