CoreWeave Opens Kimi K3 Deployments on GB300 NVL72 Dedicated Inference

How deployment works

According to CoreWeave, running Kimi K3 follows the same process as any other custom deployment on Dedicated Inference. Customers upload the weights to CoreWeave AI Object Storage, then create a gateway and a deployment that points to them, choosing the GPU class and the runtime. CoreWeave provisions a tenant-isolated gateway that handles authentication, load balancing and routing, and schedules the deployment across the selected GPUs. The endpoint supports min/max replica autoscaling.

Billing is per GPU-hour for the chosen GPU class, with no egress or ingress fees on top. The step-by-step guide in CoreWeave's technical documentation covers these steps:

• Stage the Kimi K3 weights and the DSpark draft model in an AI Object Storage bucket.
• Grant Dedicated Inference access to the bucket.
• Create a gateway to expose the model.
• Create the deployment with the validated Kimi K3 engine configuration.
• Wait for the weights to load and the engine to start.
• Send an inference request to verify the endpoint.
• Clean up the gateway and deployment when finished.

CoreWeave states that it provides neither Kimi K3 weights nor a license. Customers must bring a valid license and should review Moonshot AI's license terms first.

The model: 2.8 trillion parameters, open source

CoreWeave describes Kimi K3 as the largest open-source model currently available. It is a 2.8 trillion-parameter mixture-of-experts model. As of August, it scores 60 on Artificial Analysis's Intelligence Index, an independent benchmark covering reasoning, coding and knowledge tasks, according to the post. It also ranks #2 on LMArena's Frontend Code Arena, behind only Claude Opus 5 Max in blind developer comparisons.

The model has a one-million-token context window, native visual understanding and an always-on reasoning mode called "thinking mode." It builds on two techniques developed by Moonshot: Kimi Delta Attention (KDA), a hybrid linear attention mechanism, and Attention Residuals (AttnRes), a replacement for standard residual connections.

GB300 versus GB200 benchmarks

CoreWeave says the challenge at production scale is raising concurrent request volume while keeping time to first token and per-user decode throughput steady, especially for agentic and multi-turn workloads that reuse long contexts. It benchmarked Kimi K3 on NVIDIA GB200 and GB300 NVL72 with 16 GPUs on each platform, at concurrency levels from 1 to 32 users. The workloads were multi-turn, tool-calling interactions with a 95% cache hit rate sustained throughout.

On GB300, throughput kept scaling through concurrency 32, reaching 198,000 TPM per GPU. GB200 delivered 64,000 TPM per GPU at that level, which CoreWeave says makes GB300 209% higher. P50 decode throughput was 133% higher on GB300. From concurrency 4 through 32, GB300 had lower p50 TTFT than GB200, with reductions of 23% at concurrency 4, 32% at 8, 18% at 16 and 97% at 32.

GB200 peaked at 149,000 TPM per GPU at concurrency 16 and then dropped sharply, which CoreWeave says made concurrency 32 unusable for latency-sensitive interactive workloads. It attributes GB300's headroom to larger HBM capacity and additional AI compute.

The serving stack

CoreWeave serves Kimi K3 on NVIDIA Dynamo at the infrastructure layer and through vLLM at the runtime layer, using optimizations from the vLLM team, Moonshot AI and NVIDIA. Each deployment spans at least 2 GB300 NVL72 nodes (8 GPUs), so one NVL72 rack hosts up to 9 replicas. Each replica is pinned to a single NVLink domain, which keeps tensor and expert-parallel traffic on the NVLink switch rather than falling back to IB across racks.

The stack also offers KV-cache aware routing, which sends repeat queries to replicas with warm prefixes. CoreWeave says this reduces token cost and helps multi-turn agentic, coding and long-context workloads. Speculative decoding (MTP or bring-your-own speculators such as Inferact/Kimi-K3-DSpark or RadixArk/Kimi-K3-DSpark) improves per-user decode TPS, and prefill/decode disaggregation lets each component scale independently.

What it means for customers

For teams that already hold a Kimi K3 license and weights, the offer is a managed endpoint without having to build the serving stack themselves. CoreWeave's pitch is that a model of this size needs real infrastructure work, not only enough GPU space. The performance figures come from CoreWeave's own benchmarks on a specific agentic workload, so results may differ for other usage patterns.