CoreWeave links inference and post-training with Forge, distillation and RL tools

Post-training begins at deployment

CoreWeave argues that shipping a model no longer ends the development cycle. Once a model serves real traffic, teams see failures, edge cases and user needs that benchmarks and pre-deployment evaluations cannot fully show. According to the company, that signal can improve both quality and cost. It may reveal that a smaller model can reproduce the behavior of a more expensive one, that more supervision would help, or that reinforcement learning is a better fit.

The difficulty, CoreWeave says, is closing the loop. Production traces have to become usable training data, post-training needs compute and orchestration, and the resulting model must be evaluated and returned to serving. The company announced the three capabilities at Fully Connected 2026. The programmable training API is in preview, with an official launch planned for later this year.

CoreWeave also advises starting with the failure mode rather than the tool. Retrieval, missing context, prompts, tool failures and serving behavior can all look like model-quality problems, and changing weights will not fix an issue elsewhere in the system. When the model does need to change, the company says the technique should follow the evidence: distillation when a stronger model already produces the desired behavior, supervised fine-tuning (SFT) when high-quality examples exist, and reinforcement learning (RL) when outcomes can be evaluated reliably but the ideal response is hard to specify.

Model Distillation and the programmable training API

CoreWeave Model Distillation is described as the managed path through that loop. Inference traffic can be captured in CoreWeave Agent Lens and turned into training data. The workflow can then relabel weak outputs, test different base models and hyperparameters, deploy candidates to Serverless Inference and compare them with the model running in production.

As an example, CoreWeave cites Method. In 2024, the company used the earlier OpenPipe product to train a Llama 3.1 8B model to navigate bank IVR systems using GPT-4o outputs. In August 2026 it revisited the workload with fresh production data, relabeled it with GPT-5.6 Sol and iterated again. According to CoreWeave, the new model won 57.5% of head-to-head evaluations against the version running in production, with the base model unchanged.

For teams that want more control, the programmable training API for Serverless Training, now in limited preview, lets users write the training loop in Python while CoreWeave runs the compute and training infrastructure. The company's example mixes RL on agent trajectories with periodic supervised learning on verified demonstrations. SFT, RL and distillation build on the same training primitives, it says.

Willow, which started with supervised learning and then built scorers for reinforcement learning, is cited as a user. Its scorers targeted behaviors such as reducing filler words, improving readability and removing self-corrected phrases. Willow CTO and Co-Founder Lawrence Liu said: "I'm insanely impressed. This model genuinely has a much better grasp on text styling."

RL rollouts on CoreWeave Inference

In reinforcement learning, inference is part of training itself. The current policy generates rollouts, training produces new weights, and those weights must return to serving quickly. CoreWeave Inference now supports RL rollouts built on NVIDIA Dynamo with vLLM support. The trainer, reward logic and environment stay where they are.

According to CoreWeave, when the trainer writes a new checkpoint it can hot-load the weights onto a live deployment without affecting in-flight requests or endpoint availability. Checkpoints load as full snapshots or incremental deltas, which the company says cuts synchronization time after the first upload. The serving layer returns training-grade metadata, including logprobs, token IDs and expert routing for MoE models.

CoreWeave worked with NVIDIA and you.com to validate the feature by post-training NVIDIA Nemotron 3.5 Lightning for web search, using SFT and RL with you.com's search tools in the loop. BrowseComp accuracy rose by 22.94%, from a 36.97% baseline to 45.45%, while average tool calls fell by 30.24%. Hot-loading checkpoints was approximately 15x faster than traditional redeploy cycles, according to the company. It attributes the gains to better search tooling, post-training refinement and harness improvements working together.

What it means for teams

The pitch is that teams running models in production can improve them from their own traffic without always migrating to a newer or larger model, and without operating the distributed systems underneath training. CoreWeave Forge is positioned as the place where these workflows meet. The results cited, from Method, Willow and the you.com collaboration, are the company's own examples rather than independent benchmarks.