NVIDIA DGX Spark Adds Official Eight-Node Clusters With 1 TB of Memory

Memory is the main gain

According to NVIDIA, a single DGX Spark provides 128 GB of unified memory. Four nodes reach 512 GB, and eight nodes reach 1 TB. That capacity covers models that previously would not start on a four-node system, including the full GLM 5.3 (753B) model from Z.ai.

NVIDIA says the full GLM 5.3 checkpoint runs across the 8x 128 GB cluster, which makes it one of the largest open models that can be served locally. The model is about 470 GB in NVFP4. In an early example, the company ran GLM 5.3 Flash across eight nodes and measured roughly 80 tokens per second in generation, which it describes as suitable for interactive local development against a frontier-class mixture-of-experts model.

For the full model, measured with MTP3 and prefill at an 8K input sequence length, NVIDIA reports prefill of 1,600 tokens per second at every tested concurrency level. Decode totals rise from 31 tokens per second at concurrency one to 51 at four and 67 at eight, while per-user decode falls from 31 to 21 and then 18.

RoCEnante: a collective backend built for Spark

Tensor-parallel inference spends a meaningful share of its time in collective operations such as all-reduce. For switch-connected clusters, NVIDIA has been evaluating a Spark-optimized backend called RoCEnante. It targets the small-message, low-concurrency, modest-rank-count pattern typical of tensor-parallel decode on local AI hardware.

The backend performs an all-to-all exchange followed by a local reduction, coordinated by a single GPU kernel. Each rank shares its input with every other rank and adds the inputs locally. This means fewer communication steps than ring all-reduce, which passes data between neighbors over several rounds. The tradeoff is more data movement, so the approach loses its edge as messages or node counts grow.

On Qwen3.8-27B-NVFP4 with speculative decoding disabled, NVIDIA reports a 4.43x decode speedup at eight nodes with RoCEnante, against 3.05x without it, which it calls a 45% improvement in scaling efficiency. At four nodes the company sees roughly 5% higher prefill and 21% higher decode throughput on NVFP4, and about 18% higher decode throughput at eight nodes on the non-quantized model.

The backend is merged into FlashInfer and falls back transparently to NCCL if its startup test fails. It works only on switch-connected topologies, because it needs direct point-to-point exchange between every node. NVIDIA stresses that it is not a replacement for NCCL, which remains the supported default and the better choice for high concurrency, larger clusters or ring topologies. The original one-shot all-reduce approach came from the community (Jason Cook/el8).

Scaling from one to eight nodes, and the topology choice

To isolate cluster scaling, NVIDIA benchmarked a dense Qwen3.8-27B (NVFP4) model without speculative decoding across one to eight nodes, using llama-benchy with 8192 input tokens and 256 output tokens at concurrency 1. Decode throughput rose from 11.86 tokens per second on one node to 52.58 on eight switch-connected nodes, a 4.43x gain. Prefill went from 2,635 to 5,105 tokens per second, a 1.94x gain.

• Two nodes, direct connect: 21.71 decode tokens per second (1.838x).
• Four nodes, physical ring: 32.82 (2.77x). Four nodes, switch with RoCEnante: 36.34 (3.06x).
• Eight nodes, physical ring: 40.11 (3.38x). Eight nodes, switch with RoCEnante: 52.58 (4.43x).

NVIDIA says the best topology depends on the workload. A physical ring delivers excellent decode throughput for a single model spread across all nodes, and a switchless ring lowers cost and simplifies setup, which the company calls attractive for enthusiasts and small deployments. A switch is needed to split a cluster into independent groups, such as two groups of four, or to run workloads that need direct any-to-any connectivity, including MPI-based clustering.

Multi-model routing and local pre-training

NVIDIA also describes NeMo Switchyard, which routes requests between model pools on a shared eight-node cluster. According to the company, it matched Qwen-only quality on a tool-use benchmark while raising aggregate throughput by up to 3x when most traffic shifts to a smaller model.

For training, NVIDIA says pre-training a nanochat model from scratch across eight nodes achieved a 7.79x speedup at 97.4% scaling efficiency. The company presents this as near-linear training performance for small models on the desk.

What it means for developers

The update targets developers who want to work with very large open models, run several models at once or experiment with distributed training without leaving local hardware. NVIDIA Sync and its Cluster Assistant handle cluster setup and device management, and support for eight-node clusters in NVIDIA Sync is coming by the end of the year, according to NVIDIA. The company points users to guides for connecting multiple Spark machines, serving LLMs with vLLM across DGX Spark, and a networking walkthrough that starts from a two-node setup.

All figures come from NVIDIA's own benchmarks, and the company notes that NCCL remains the better option for high concurrency and larger clusters, so results will depend on topology and workload.