Vapi adds traffic splitting so teams can test voice agent changes on live calls

What traffic splitting changes

According to Vapi, teams could already create versions of an assistant, but publishing one moved all calls to it at once. That left builders with no reliable way to confirm that a changed prompt, model or voice worked as intended with live customers, or whether it beat the previous version.

The company says this made teams cautious. In its words, teams "got careful, then slow, and eventually they stopped iterating once things were good enough." With the new feature, a team can send 0.1%, 1% or 5% of calls to a new version, watch the metrics, and then roll forward or back. Vapi describes it as "essentially a canary for your assistant."

Peter Gomez, a Member of the Technical Staff at Vapi who helped build the feature, argues that the risk is greatest with LLMs because the outcome of a change is not deterministic. Adding a single keyword to a prompt, he says, can have downstream effects nobody planned for. He adds that simulations and evals get teams to maybe 90% confidence before launch, and that real calls cover the rest.

How it works and how teams use it

Gomez says the hardest engineering problem was changing code that sits in the hot path, where every call touches it whether or not the feature is used. Vapi put everything behind feature flags, ran extensive internal testing and made rollback easy.

The second challenge was what Vapi calls stickiness: keeping repeat callers on the same version. The team had to identify a caller across phone calls, web calls, inbound and outbound, so that someone who calls back gets the same version wherever possible. Identity can be switched on or off for deployments where this is required.

Before the launch, according to Gomez, some teams ran duplicate assistants and routed between them, some versioned through Vapi's GitOps repository, and others built their own routing logic using the assistant request as an escape hatch. He calls most of these solutions more complex and less reliable than the built-in option.

Looking at customer data, Vapi found that splits are usually temporary. Once a team trusts a new version, it often sets the assistant back to "follow latest" so new versions go straight to all traffic. Gomez describes the pattern as "split, confirm, flip back, repeat." Swapping the LLM gets a split, while a small prompt tweak typically goes to 100% and is checked a few hours later.

Best practices and what comes next

Vapi recommends starting with two-way splits. The feature supports more, but two at a time makes it clear what is driving a difference. Because every call is pinned to its version, results can be attributed cleanly, and many experiments reach statistical significance quickly. Some changes, particularly with lower volume or spiky traffic, may need a week or more. Teams that maintain duplicate assistants or custom routing only to avoid the all-or-nothing publish are told to consider simplifying.

Gomez says early users treat the feature as a gate rather than a long-running experiment tool. He also notes that hesitation to adopt came from teams who had already engineered their own workaround, since replacing a working in-house solution takes a deliberate decision.

On the roadmap, Gomez calls traffic splitting a primitive for outcome-based features. With calls pinned to versions, automatic rollouts could become feasible alongside statistical experiments. His example: if a structured output measures deflection rate, Vapi could keep a version only if deflection holds and roll back if it drops. He says the company is still evaluating this direction.

For builders on Vapi, the practical effect is that trying a new model, prompt or voice no longer requires exposing the whole customer base. Setup instructions are in Vapi's documentation.