Modal Says NVIDIA Vera CPUs Can Double Sandbox Throughput in Early Tests

A first Arm server for a long-time x86 shop

According to Modal, NVIDIA hand-delivered the company's first Vera CPU server to its San Francisco office last week. Vera is NVIDIA's new Arm-compatible CPU. It was built for the Vera Rubin GPU platform but is also available as a standalone CPU server.

The move would be a big change for Modal, which has a long history of running exclusively x86 machines. The company says it would need to change its infrastructure and its image build system to support a different CPU architecture. Despite that work, Modal calls the benchmark results impressive and describes Vera as an "incredibly compelling option" for powering Modal Sandboxes.

Why Sandboxes put unusual pressure on CPUs

Modal explains that orchestrating a Sandbox product means running hundreds of concurrent containers and VMs on each cloud host. Workloads are unpredictable in several ways:

• Resource usage can be extremely spiky. A sandbox may sit mostly idle waiting on an LLM, then surge to 10x its baseline CPU use because of an npm install or an expensive grep command.
• Workload shapes are not known in advance, because an LLM can put anything into the sandbox.
• Resource sizing is often a guess.

Because these mixed workloads share one pool of compute, noisy neighbor problems can arise, where one tenant's behavior affects another's performance. Modal Sandboxes support bursting, which lets users reserve guaranteed capacity for typical P50 usage instead of the worst-case P100 moment. Modal says this has saved users millions of dollars, but it increases exposure to noisy neighbor issues.

CPU time also carries wider costs, the company argues. For background agents, such as Ramp Inspect or those running on Niteshift, faster agents mean faster answers and lower CPU time. For reinforcement learning, GPUs sit idle while sandboxed processes run, and most training setups are still synchronous, so any time saved on CPU is time saved on GPU.

The benchmark: Agentless Agent Bench

To test Vera, Modal built Agentless Agent Bench, a collection of 50 Terminal-Bench oracle solutions and verifiers. They are stripped of all network calls and prewarmed in concurrent Docker containers, so that only the work done on the CPU remains, such as compilation, file grepping, git operations and device emulation.

In the first test, run serially on a machine without load, Modal's author says he was skeptical. Across the 50 tasks, Vera often performed better than the other CPUs, but not always, and not by a large margin.

Under load, the picture changed. The throughput test raises the number of concurrent containers and loops as many task completions as possible in a fixed time. Modal reports up to double the task throughput on Vera compared with an equivalent x86 GNR machine, while individual task latency was equal to, or sometimes lower than, the x86 machine. According to Modal, Vera does not slow down as jobs accumulate. The company says the Vera team confirmed the results were not surprising.

What it could mean for Sandbox users

Modal attributes Vera's performance under contention to its high memory bandwidth, which it says allows significantly more container workloads per core without CPU throttling. As CPU counts rose, the alternatives hit early throughput plateaus while Vera kept improving. Modal suggests it could push Vera further than the node it had for benchmarking.

For Modal, this means greater packing density and more forgiving burst protection. For Sandbox users, the company says it can offer higher absolute scale and throughput without sacrificing latency. Modal says it will roll out support for Vera CPUs for Sandboxes and work with NVIDIA and cloud providers to scale up its fleet. The results are early and come from Modal's own benchmark, and the source gives no timeline or pricing.