Fiddler's Centor adds Dynamic Model Escalation to cut false positives in AI safety evals

The trade-off Fiddler wants to remove

Fiddler argues that teams evaluating AI agents at scale usually get only two of three things: accuracy, speed or low cost. In its announcement, the company lists the options teams face today. They can send data to an external model at OpenAI or Anthropic for a sharper evaluation, at the risk of fast-growing costs. They can sample a fraction of inputs to hold down spending and hope that what slips through does not become a reputational problem. Or they can run slow evals that either delay the end user's response or let a bad output through before it is flagged.

Fiddler says every input and output deserves 100% trace coverage without these compromises. Centor is the product it built for that purpose, and Dynamic Model Escalation is the latest capability added to it.

How Dynamic Model Escalation works

Centor routes each input to the model best suited to evaluate it, either a Centor Model or a general-purpose model, without sharing data outside the customer's environment. The Centor Models are task-specific, real-time specialist evaluators behind Safety, Faithfulness and Sensitive Information scoring. According to Fiddler, they handle the vast majority of traffic quickly and with no per-call charge from a third-party provider.

Dynamic Model Escalation is now available for the Centor Model for Safety. When an input falls outside the domain the small specialist model was trained on, it is automatically handed to a Fiddler-hosted general-purpose model for a second evaluation. This happens inside Centor, with no external API call.

Detecting such inputs does not need an extra model or additional computation, Fiddler says. The Centor Model for Safety uses the embeddings it already produces while scoring and looks at how far an input sits from its training data. That signal can trigger an escalation.

• The feature is optional. Customers can switch it on for added accuracy or keep scoring with the Centor Model for Safety alone.
• Only the escalated share of inputs takes on added latency. Fiddler says the rest run at the same speed as today.
• On Fiddler's SaaS-hosted Centor, escalation comes at no added per-eval cost. For self-hosted customers, running the general-purpose model adds to their own infrastructure cost.

The reported result

Fiddler cites a test from development and production testing of the feature, in which a customer ran data through the Centor Model for Safety. By escalating just 3.1% of inputs, the company says it reduced the overall jailbreak false positive rate by 93.5%, which it describes as drastically cutting the triage queue for security teams. The announcement presents this as an example result on one reference dataset, not a general guarantee, and the figures are Fiddler's own.

Fiddler is also careful about what the general-purpose model is. It says the model is not better than the Centor Models. It is a generalist suited to inputs outside a Centor Model's trained scope, while the Centor Models remain faster and more accurate within that scope.

What it means for AI teams

For teams already using the Centor Model for Safety, the feature offers a way to improve accuracy on unfamiliar inputs without moving data to an outside provider and without adding a per-call fee on Fiddler's SaaS offering. Because it is optional, teams can compare results with and without it. Self-hosted customers should plan for the extra infrastructure needed to run the general-purpose model. Fiddler invites interested teams to request a demo to see how their agentic applications would benefit.