CoreWeave: Inference, Not Training, Is Now the Defining Layer of AI
CoreWeave argues that the center of gravity in AI has moved from training to inference, citing Futurum Group projections that the data center AI semiconductor market for inference will reach $885 billion by 2030. In a blog post, the cloud provider says running models in production is a system-level challenge and presents its own full-stack platform as the answer.
- Futurum projects the inference data center AI semiconductor market will reach $885 billion by 2030, growing 7.4x versus 2.7x for training.
- Agentic and reasoning inference is projected to grow 219% in 2026, reaching $546 billion by 2030.
- Per the post, six in ten enterprises wait over four months from buying GPUs to a first production request.
- CoreWeave says production inference is a system problem and promotes a full-stack platform from Serverless Inference to Inference on CKS.
- CoreWeave says it was first to validate NVIDIA Vera Rubin NVL72.
Inference market outgrows training
According to the post, The Futurum Group projects that inference semiconductor spending in data centers will grow 7.4x over five years, against 2.7x for training. CoreWeave says the questions teams ask have changed as a result. Two years ago, teams asked whether a model was smart enough, how it scored on benchmarks and how many parameters it had. Once a model serves real users, the questions become operational: is it up, does it respond quickly, does it return the right answer, and what does it cost to run?
The post cites Futurum's 2026 survey of 820 AI decision-makers, in which 68% of enterprises were already past experimentation, in the optimization, standardization or transformation stages. Accuracy and availability top what enterprises measure, while benchmark scores and parameter counts are nowhere near the top of the list, according to the post.
Agents multiply the load
CoreWeave calls agentic AI the fastest-moving part of inference. Futurum projects agentic and reasoning inference will grow 219% year over year in 2026, from roughly $36 billion last year to $546 billion by 2030. By the end of the decade, the post says, agentic inference alone is larger than the entire training market. It adds that more than 40% of enterprises already run some form of agent in production.
The complexity differs from a chatbot, which answers and releases the GPU. An agent thinks in steps, calls tools, retrieves information and may start other agents in parallel, so one request can trigger dozens or hundreds of model calls. CoreWeave lists the resulting problems:
- The GPU stays reserved across the whole chain.
- The I/O pattern is unpredictable.
- One task can generate 10 to 100 times more tokens than a normal query.
- A single slow step stalls everything downstream.
- Multiple coordinating agents do all of this at once.
Security remains the number-one concern at every maturity stage, per the survey cited, and teams furthest along in deployment were more worried about it, not less.
Capacity delays and system failures
The post says six in ten enterprises wait more than four months from buying GPU capacity to serving a first production request. The most common range is four to six months, and only 6% manage it in under two months. A third of enterprises name accelerator availability as their biggest barrier to expanding AI infrastructure; with memory and storage constraints, it is more than half. CoreWeave notes that specialist AI cloud providers are projected to grow 8.3 times over five years, against 4.5 times for hyperscalers.
CoreWeave also argues that benchmarks mislead. They measure isolated runs under ideal conditions, while production brings concurrent users, variable input lengths, cold starts and noisy neighbors. The company names four common failure points: cold starts, sudden traffic spikes, observability gaps and cost drift as agentic workloads consume far more tokens.
CoreWeave's full-stack pitch
CoreWeave describes a "metal to agent" approach. Its stack includes bare-metal access to the latest NVIDIA GPUs, high-speed networking for multi-node serving, and CoreWeave AI Object Storage with local GPU cache to cut cold starts. The serving stack is tuned for KV cache management, batching, autoscaling and routing, and CoreWeave Mission Control covers security, observability and operational control.
The company says workloads can move from Serverless Inference to Dedicated Inference and then to Inference on CKS without replatforming. Interfaces are framework-agnostic and open-source-compatible, and direct engineering support is included, according to the post.
CoreWeave says it was the first cloud provider to make NVIDIA GB200 NVL72 generally available, the first to deploy NVIDIA GB300 NVL72, and now the first to bring up and validate NVIDIA Vera Rubin NVL72. It also cites leading MLPerf Inference v6.0 performance for DeepSeek-R1 on GB200 and GB300 NVL72, and a second consecutive ClusterMAX Platinum rating from SemiAnalysis.
What it means for teams
The post is a vendor's argument, but its core point is that choosing a model is only the start. Teams moving from prototype to production face capacity lead times, latency under live traffic, security isolation and token costs. CoreWeave says the line between training and inference is also blurring, as models improve through reinforcement learning and fine-tuning on production traffic, so infrastructure needs to handle both.
