Apple: Four Mac Studios Run Nearly Trillion-Parameter Muse Spark Open at 36+ Tokens/s

Muse Spark Open on a desktop cluster

In a post on its machine learning research site, Apple describes how researchers and developers can run frontier-class models locally on Mac hardware with Apple silicon. According to Apple, MLX uses the unified memory architecture of Apple silicon so that people can run new inference and fine-tuning methods on their own machines in a private environment.

The headline example is Muse Spark Open, which Apple describes as having nearly a trillion parameters. On a cluster of four Mac Studios, the model generates over 36 tokens per second, which Apple calls "well above average human reading speed". The figure was measured on Mac Studio with M5 Ultra at a context size of 8,192 tokens. Apple released a new version of MLX alongside the post.

Muse Spark Open is a mixture of experts model with 816B total parameters and 42B active parameters. Apple evaluated it in 4-bit NVFP4 precision, while its native precision is BF16.

Single machine results: M5 Ultra versus M3 Ultra

Apple benchmarked three models with the MLX LM package: Muse Spark Open (816B), Mistral Medium 3.5 (128B) and Qwen 3.8 (27B). Mistral Medium 3.5 and Qwen 3.8 are dense models, evaluated in FP8 and BF16 respectively. The company measured time to first token, generation throughput and peak power, at context sizes of 8,192 and 65,536 tokens. Apple notes that agentic uses, such as coding applications, often need large contexts.

Qwen 3.8 and Mistral Medium 3.5 fit on one Mac Studio with 512GB of memory, taking up to 68GB and 158GB of RAM at the 65,536-token context size. Apple compared the M5 Ultra with the previous M3 Ultra:

• Time to first token: up to 4.6× faster on the M5 Ultra, which Apple attributes to the GPU Neural Accelerator and its dedicated matrix multiplication support.
• Generation speed: 1.6× faster tokens per second, which Apple links to memory bandwidth of 1.2 TB/s, up from 819 GB/s on the M3 Ultra.
• Peak power: about 200W for the M3 Ultra and 300W for the M5 Ultra.

Apple argues that this modest power draw means a Mac Studio can be deployed for AI work in any home or office.

Splitting a model across several Macs

Muse Spark Open does not fit on one machine. According to Apple, it would need more than 1.6TB of RAM for the parameters alone in BF16. In NVFP4 the footprint falls to about 493GB, but with the memory needed for intermediate computations during inference it still cannot fit on a single Mac Studio.

Apple describes two ways MLX splits a model. Pipeline parallelization divides it by depth: one Mac Studio runs the first half of the layers and passes its output to a second machine, which runs the rest. Apple says this is simple to implement and needs only minimal code changes. Tensor parallelization divides the model by width. Figure 1 in the post reports results for pipeline parallelization over two Mac Studios, and tensor parallelization over two and four.

Both approaches depend on fast links between machines. Apple says macOS 26.2 introduced a low-latency RDMA driver over Thunderbolt 5. MLX version 0.30.1 and later builds on it and adds communication collectives called jaccl. In Apple's setup, two Thunderbolt cables connect each pair of Macs to double link bandwidth. The mlx.launch command starts jobs across multiple nodes, and an experimental flag, MLX_METAL_FAST_SYNCH=1, reduces overhead between the CPU and GPU.

Apple says decode speed with pipeline parallelization is very similar to a single machine, since layers are still evaluated one after another, and communication overhead is low because only intermediate outputs are exchanged.

What it means for local AI work

According to Apple, the results put models of nearly a trillion parameters within reach of a desk-based setup, with data staying on local hardware. Apple positions the M5 Ultra Mac Studio as the desktop of choice for tasks with very large contexts, and says multiple Macs linked over Thunderbolt can serve models too large for one machine. The figures come from Apple's own benchmarks using mlx_lm.benchmark, and the post does not state pricing for the hardware.