Scale AI Urges Washington to Build Its Own Frontier AI Testing Capacity
Scale AI is calling on the US government to build its own capacity to test frontier AI models, arguing that policy should rest on evidence rather than speculation. In a blog post dated September 25, 2026, Scale's Francis deSouza says Washington's ability to evaluate models has not kept pace with the companies building them.
- Scale AI's Francis deSouza says the US government needs its own independent testing of frontier AI models.
- He proposes a clear mandate, pre-deployment evaluation and dedicated, sustained funding.
- The President requested $27 million for CAISI in fiscal year 2027, while House appropriators proposed up to $15 million.
- Scale says most tested agents were vulnerable to injected instructions in clarification answers, with attacks about 10 times more frequent for several models.
- In a cyber defense competition, AI models refused to help defenders 12 percent of the time, and over 40 percent when asked how to lock down systems.
Evidence over speculation
DeSouza writes that much of the current debate about AI risk rests on speculation, and that some of the loudest warnings have "little grounding in science" yet still drive policy. He argues that errors cost in both directions: moving recklessly would be a mistake, but slowing down out of fear would be just as wrong. According to the post, every week lost means slower progress toward new cancer treatments, stronger American national security and scientific breakthroughs.
The post ties the argument to this week's UN General Assembly, where AI is a central topic as world leaders discuss international coordination. DeSouza says any agreement will only be as strong as the evidence beneath it, and that governments need to be specific about benchmarks, who verifies results, and what happens when serious risks are found.
Three requirements for a government testing system
DeSouza notes that most frontier model testing today happens inside the labs themselves. He says these companies have a moral responsibility and a strong commercial incentive to release safe products, but on national security and public safety, Washington "cannot let the labs grade their own homework." The government needs its own evidence on cyber, biological, autonomy-related and other emerging risks, he writes.
The post lists three things an effective system requires:
- A clear mandate: an accountable official to coordinate testing across federal agencies, set government-wide evaluation priorities, mobilize expertise in the public and private sectors, and ensure serious risks are identified and mitigated.
- Pre-deployment evaluation: a stronger ability to evaluate frontier models before deployment, by working with developers to secure access for testing and reassessing national security risks as capabilities evolve.
- Dedicated capacity: sustained government funding for independent public-private testing, including tools, resources and technical expertise across national security domains.
On funding, the post says the President requested $27 million for CAISI, the government's frontier AI testing center, in fiscal year 2027, while House appropriators proposed up to $15 million. DeSouza calls these levels like managing JFK air traffic control from a small-town control tower.
What Scale says its research shows
Scale says it has spent years building expertise and infrastructure to test advanced models. According to the company, it has partnered with government AI evaluation bodies in the United States, the United Kingdom, Korea and Singapore to develop benchmarks and expand testing capacity, with more to share in the coming weeks. It also says it assists national security partners in building cyber evaluations that measure what models can do against real-world threats.
The post highlights two findings from Scale's cyber research. The first concerns AI agents that handle tasks such as email and banking. Scale's researchers found that when an agent pauses to ask for clarification, an attacker can slip harmful instructions into the answer. Most models tested were vulnerable. For several models, attacks succeeded about 10 times more often when malicious instructions appeared in answers to their questions than when hidden in emails, documents or webpages they read. Such attacks can also go unnoticed, because the agent can complete the user's task correctly while following the attacker's instructions.
The second finding runs the other way. In a national cyber defense competition where students defended live systems against professional attackers, AI models refused to help the defenders 12 percent of the time, and more than 40 percent of the time when asked how to lock down their systems.
What it means for policymakers
DeSouza argues that without testing built for such scenarios, policymakers cannot know what banks, power grids and hospitals need to be ready for. With the capacity to direct outside experts and act on their findings, he says, the government can address real risks before they become crises. The post closes with the line: "The nation that can measure the frontier will be the one that steers it."
The piece is an advocacy statement from a company that sells evaluation services, and its claims about research results and partnerships are Scale's own.
