CoreWeave posts 800 TPS on Llama 3.1 405B in MLPerf with NVIDIA GB200
CoreWeave says it is the first cloud service provider to submit MLPerf Inference v5.0 results for NVIDIA GB200 Superchips, reaching 800 tokens per second (TPS) on the Llama 3.1 405B model. The company also reported 33,000 TPS on Llama 2 70B using NVIDIA H200 GPU instances.
- CoreWeave says it is the first cloud provider to submit MLPerf Inference v5.0 results for NVIDIA GB200 Superchips.
- A GB200 instance with two Grace CPUs and four Blackwell GPUs delivered 800 TPS on Llama 3.1 405B.
- H200 instances reached 33,000 TPS on Llama 2 70B, a 40 percent throughput improvement over H100 instances, according to CoreWeave.
- The company says it was first to offer general availability of NVIDIA GB200 NVL72-based instances.
GB200 result on one of the largest open-source models
According to CoreWeave, its MLPerf v5.0 results set a new industry benchmark in AI inference. The company used an instance built on NVIDIA GB200 Grace Blackwell Superchips, which features two NVIDIA Grace CPUs and four NVIDIA Blackwell GPUs. On that configuration, CoreWeave delivered 800 TPS on the Llama 3.1 405B model, which it describes as one of the largest open-source models.
CoreWeave states that it is the first cloud service provider to submit MLPerf Inference v5.0 results for NVIDIA GB200 Superchips. The figure is listed in MLCommons entry 5.0-0076, according to the footnotes in the release.
Peter Salanki, Chief Technology Officer at CoreWeave, said the company is "committed to delivering cutting-edge infrastructure optimized for large-model inference through our purpose-built cloud platform." He added that the results "reinforce CoreWeave's position as a preferred cloud provider for leading AI labs and enterprises."
H200 instances show 40 percent throughput gain
CoreWeave also submitted new results for NVIDIA H200 GPU instances. It reports 33,000 TPS on the Llama 2 70B model, which the company describes as a 40 percent improvement in throughput over NVIDIA H100 instances. This result corresponds to MLCommons entry 5.0-0077, according to the release.
The company says these numbers further show it as an industry-leading cloud infrastructure services provider.
Track record with NVIDIA hardware and the benchmark itself
CoreWeave notes that this year it became the first to offer general availability of NVIDIA GB200 NVL72-based instances. Last year, it was among the first to offer NVIDIA H100 and H200 GPUs, and one of the first to demo NVIDIA GB200 NVL72.
MLPerf Inference is described in the release as an industry-standard suite for measuring machine learning performance across realistic deployment scenarios. The company points out that how quickly systems can process inputs and produce results with a trained model has a direct effect on user experience.
CoreWeave, which calls itself the AI Hyperscaler, was established in 2017 and completed its public listing on Nasdaq (CRWV) in March 2025.
What it means for customers
For teams that run large models, the results give a standardized reference point for inference speed on GB200 and H200 cloud instances. CoreWeave positions the numbers as evidence that its platform suits AI labs, startups and enterprises that serve large models. The release does not provide pricing or other performance figures beyond those listed above, so comparisons with other providers would need to rely on the MLCommons entries cited by the company.
