MLCommons Launches Candela to Test AI Safety and Jailbreak Risks

What Candela offers

The two benchmarks measure, respectively, how readily an AI system produces harmful content and how easily attackers can trick it into doing so. Both now run on demand in Candela, according to MLCommons.

Candela users can run practice tests and receive a report with their Safety and Jailbreak percentiles and raw scores for internal use and product improvement. Annotated Practice results can also be requested. These include the practice prompts, responses and annotations behind a score. Official tests allow users to make public claims about their scores, in line with MLCommons benchmark use and messaging guidelines.

MLCommons says the service tests the system as it is actually deployed. A small Candela client relays each prompt to the production system, with its system prompt, guardrails and retrieval in place. The client returns each answer for evaluation, and nothing else: no keys, no weights, no configuration details.

Safety 1.2.1 and the first public results

The Safety benchmark tests how reliably a text-based AI system refuses or safely redirects requests that could lead to harm. Its prompts cover the 12 hazard categories in the AILuminate Assessment Standard, from violent and nonviolent crimes and indiscriminate weapons to hate, privacy, defamation, and specialized advice on health, law and finance. MLCommons says the standard focuses on everyday failures in widely deployed products rather than speculative concerns, and points to real incidents such as a Canadian tribunal holding an airline liable for what its customer-service chatbot told a customer.

Version 1.2.1 is a minor scoring update that aligns Safety with Jailbreak 1.1. Earlier changes in the 1.2 line include a deterministic evaluator trained by MLCommons in place of LLMs acting as judges, a new official prompt set called Problem Set 2.0 with 3,900 prompts, and scores expressed as percentiles from 0 to 100 on a reference scale.

Public results cover 27 systems: 22 open-weight models from developers including Google, IBM, NVIDIA and OpenAI, plus 5 guardrail systems. The open-weight models score from the 4th to 98th percentile. MLCommons says this shows that the choice of model alone can move a deployment's relative safety profile from near the bottom of the reference distribution to near the top. Guardrail systems, which screen what a model receives or returns, score at or near 100.

Jailbreak 1.1 and a new evaluator

The Jailbreak benchmark takes the same hazardous prompts as Safety and rewrites each using known attack techniques. Because both benchmarks judge responses against the same standard, comparing a system's two scores shows how much baseline safety an attacker could strip away with published techniques. The MLCommons Jailbreak Taxonomy groups attacks into four families by mechanism, including role-play and template attacks and wrapper and schema attacks.

In Jailbreak 1.0, across a limited set of systems, violating responses rose from 11.08% on plain prompts to 18.65% under attack. Role-play and wrapper attacks were the most effective, with mean violation rates of 35.7% and 32.8%.

Jailbreak 1.1 public results are not yet out. MLCommons says they will follow once its notification period for developers, already underway, ends. The organization argues that naming a highly jailbreakable model is itself an information hazard, and that the benchmark includes only published jailbreaks.

Member feedback on the June private release of Jailbreak 1.0 found the automated evaluator's error rate too high, because attacks meant to confuse the tested systems also confused the LLM judges. MLCommons built its own deterministic classifier in response. It says the evaluator agrees much more often with human judgment, and that no LLM judges its creator's other models or its competitors' models. Jailbreak 1.1 runs on it with largely the same attacks as 1.0.

What it means for AI builders and buyers

According to MLCommons, the custom evaluator and a more discriminating prompt set cut the time from starting a Safety test to getting results from about 3 hours to 1 hour, while improving evaluation quality.

For teams deploying AI, Candela offers a way to measure a production system and compare it directly with current popular models, using percentile scores that can be cited publicly after an official test. The Jailbreak results will add the attack-resistance side once the developer notification period finishes.