MLCommons: Base Model Safety Scores Don't Carry Over to Deployed AI Systems
MLCommons says AI safety evaluation should stop assuming that a base model's safety score still applies once teams add system prompts, retrieval layers or fine-tuning. In a post dated 29 October 2026, the organization argues that deployed systems need their own testing, and points to its new Candela benchmarking service.
- MLCommons argues AI safety results are not composable: a base model's score cannot be assumed to hold for a deployed system.
- Research from CDT and MIT found fine-tuned medical and legal models scored both safer and less safe than their base models, with no predictable pattern.
- Candela, a new benchmarking-as-a-service product, tests the full deployed assembly in the customer's own tenant.
- AILuminate Safety and Jailbreak releases in November will include margins of error, according to MLCommons.
The composability problem
The post, written by Bennett Hillenbrand, starts from a contradiction in practice. Practitioners know a deployed system behaves differently from its base model, and that its behavior keeps changing as they alter the system prompt, retrieval layers or fine-tuning. Yet safety evaluation often assumes the opposite, with teams citing the base model's safety score from the system card for whatever they have built on top.
MLCommons uses the term composability for how well evaluation results for an upstream system carry over to the systems built on it. According to the post, this assumption generally held in classical software. In AI safety, it often has not, with proper testing showing otherwise.
Scores move in both directions
The post cites research from CDT's AI Governance Lab and MIT's Algorithmic Alignment Group, which compared publicly available medical and legal fine-tunes with their base models. Some fine-tuned models scored as safer than their base models and some as less safe. No single fine-tuning choice predicted which direction a model would move, and the amount of modification did not predict the amount of change in safety either.
Changes can also improve results. The post notes that adding runtime guardrails to an open-weight model generally improves safety scores, though how much is often unclear without proper testing. It also references the paper "Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!" as an example of changes that worsen safety.
Noisy benchmarks and the Candela response
A second obstacle is measurement. Citing a recent paper by Solomon Messing, the post says many widely used benchmarks are too noisy to reliably compare pre- and post-deployment systems, track shifts over time, or compare one system with another. Noise can come from which test prompts were selected, how they were worded and which evaluator graded the responses. Without normalization, a score change may reflect measurement noise rather than a real change in safety. The post discloses that Messing was employed by MLCommons when he wrote the paper and that the author is acknowledged in it.
MLCommons says its new Candela benchmarking-as-a-service product brings AILuminate measurement to the full production assembly: the base model together with its system prompt, safeguards, retrieval, tools and filters. It is described as a private test run in the customer's own tenant.
The organization also points to recent changes to its evaluator, including classifiers instead of LLMs-as-judges and Item Response Theory (IRT) methodology for benchmark content development. According to MLCommons, these allow it to generate defensible margins of error. The next releases of AILuminate Safety and Jailbreak in November will include margins of error that account for variance across the evaluation pipeline, with transparency about which sources of variance are covered. MLCommons says that to its knowledge no other widely used AI safety benchmark reports margins of error this way.
What it could mean for deployers and regulators
With known measurement uncertainty, the post argues, researchers could test which modifications preserve a safety result, which break it, and whether either outcome can be predicted in advance. Today, it says, the defensible position rests on observation that results do not carry over given certain changes.
If some changes are known to preserve a score, teams could decide which to deploy untested, which to give a cheaper smoke test and which to scrutinize fully with an annotated report. If a base model's score predicts downstream scores, upstream results could guide model choice and show how much a deployment can change before those scores stop applying. MLCommons adds that such scoring could help regulators draw lines between a developer's and a deployer's obligations, and reduce measurement and compliance burdens.
Teams deploying AI can test their systems with Candela now. Researchers interested in composability or measurement error can join the AIRR Analytics Working Group, whose parent AIRR working groups draw participants from more than 80 organizations, according to MLCommons.
