Scale AI Outlines Four-Phase Testing Lifecycle for Frontier AI Models

A lifecycle instead of a one-off test

The post follows a recent call by Scale CEO Francis de Souza for Washington to "Steer the Frontier" by strengthening federal testing and evaluation (T&E) capabilities. According to Scale, building such an ecosystem requires a clear mandate for government officials who can effect change, a commitment to pre-deployment evaluation of highly capable models, and dedicated, adequately funded capacity.

Scale says that testing by frontier labs of their own models is irreplaceable, but that it cannot be the only way the U.S. government understands the technology. T&E, it argues, is not one-size-fits-all and cannot be a point-in-time measurement. Mixing up the distinct phases, the company warns, risks misunderstanding what each one contributes to assessing and reducing model risk.

The four phases

Scale describes four stages, each run by different actors and measuring different things:

• R&D testing: frontier labs test internally to understand new model behavior and capabilities. This stage is governed by voluntary private frameworks, and Scale points to Anthropic's "Frontier Safety Roadmap" and Meta's "Advanced AI Scaling Framework" as examples.
• Pre-release model testing: the U.S. government tests models after post-training but before release, to assess national security implications and to avoid taking labs' claims on faith.
• Pre-deployment system testing: the organization putting a model into its own products or operations tests the whole AI system across use cases.
• Post-deployment testing: industry and government keep testing models and systems in production.

Scale says each phase answers a different part of the same questions: does the model perform as expected, is it as capable as believed, and is it as robust as assumed.

Where the gaps are, according to Scale

On R&D testing, Scale notes that it helps labs build an evidence-based view of where models succeed and fail and whether safeguards are needed. It cites OpenAI's models breaching Hugging Face as an example of an event at this stage, and the company's public vow to pause training and testing of certain models. The recent "White House Accord on Super Intelligence" focuses almost entirely on this stage, according to the post, calling for internal controls, internal monitoring teams, partnerships with external evaluators and a board-level oversight committee at each company.

For pre-release testing, Scale says labs are in a sense "grading their own homework", which is why it supports stronger independent government evaluation. The post states that no U.S. federal law currently stops frontier models from being released without government pre-review or testing. Executive Order 14409, a voluntary framework issued in June by the Trump Administration, asks labs to work with the government on testing highly capable cybersecurity models. Scale adds that all major U.S. frontier labs have signed agreements allowing government testing of their models more generally, most of them before the June order. The problem, it says, is that the expertise to test rigorously and in depth rarely exists inside government today, so officials will need to work closely with independent experts and organizations on evaluation frameworks and review thresholds.

Systems and production use

The third stage concerns companies or governments that integrate AI into decision-making, agent workflows, customer-facing services or applications. Scale says a model that looked safe in earlier stages can behave differently once connected to tools, memory, file permissions and other models. Regulated industries face compliance requirements, such as the Securities and Exchange Commission's rules for financial services, and may self-report or work with external auditors.

The fourth stage addresses behavior over time. Scale notes that behavior may "drift" from the original standard of alignment, users may supply unexpected inputs, and an attacker who fails a single jailbreak attempt may succeed by escalating gradually over a long conversation, a technique called "multi-turn" jailbreaking. Labs, independent third-party testers, governments and enterprises all have a role in catching real-world failures.

What it means for industry and policy

Scale argues that a strong testing ecosystem is compatible with innovation. In its view, the clearer and more consistent the government's expectations, the more confidently industry can invest in and deploy advanced models. The company frames the push against recent concerns about agents breaking out of sandboxes and the potential of self-improving AI, and says understanding models across all life stages is more important than ever. The post is a statement of Scale's policy position rather than a government proposal, and it leaves open how a federal mandate and funding would be set up.