AI Now Institute Challenges the Direction of the AI Safety Debate

A dispute over what AI safety means

The article argues that comparisons between artificial intelligence and nuclear technology have created confusion about how AI risks should be managed. The institute says it finds those comparisons frustrating because people who make them are also denying that standard risk assessment techniques can be sufficient to control AI.

According to the AI Now Institute, risk assessment methods were developed in part through work connected to the Manhattan Project. Scientists concerned about potential harm created rigorous standards for systems that could affect human life, the environment, financial assets and national security assets.

The question of how safe is safe enough

The institute presents “How safe is safe enough?” as the central question for safety engineering. It describes that question as democratic because it concerns the level of risk that society is prepared to accept, rather than leaving the decision solely to technical specialists or private companies.

In the article’s account, established safety work is built around methods for assessing possible harm and setting limits. The institute says this approach provides a way to discuss AI safety using standards and decisions that can be examined rather than relying on broad claims about hypothetical future dangers.

Criticism of large AI labs

The AI Now Institute says large AI labs are “renegotiating these safety thresholds through the co-option of terminology.” It argues that this is happening under the pretense that AI safety should instead focus on what it calls an “unsubstantiated, unscientific, and hypothetical concern.”

The institute also criticizes AI engineers who, in its words, claim that they get to determine what AI safety is and where its thresholds should be drawn. It describes the resulting dispute as “a religious argument” and says that it is “very anti-democratic.”

What the argument means for AI governance

The article’s message is that AI safety should be treated as a public decision about acceptable harm, supported by rigorous risk assessment. That would place questions about safety thresholds in a broader democratic debate, rather than allowing large AI labs or engineers to define those limits by themselves.

The AI Now Institute published the argument as part of an article titled “The AI Safety Debate Has Lost the Plot. We Asked 10 Experts What the Real Nightmares Are,” dated November 10, 2026. The institute directs readers to the full article for the wider discussion.