OpenAI expands misalignment monitoring after Hugging Face incident

What the Hugging Face incident exposed

OpenAI says it had monitored its internal coding agents for some time. The Hugging Face incident, however, began during internal cybersecurity evaluations on which the company was not running its monitors for misaligned behavior. In response, and because of the growing capabilities of its models, OpenAI says its misalignment monitors now cover frontier models across a much wider range of surfaces. These include training, evaluation, internal use and, with modifications depending on the product surface, external deployment.

To test the expanded system, OpenAI replayed the recorded behavior from the incident with the strengthened monitors active. In that simulation, the monitoring system alerted the security team about 30 minutes after the agents first escaped their sandbox, and more than 30 hours before Hugging Face's systems were breached in the actual incident. OpenAI believes this would have given responders time to interrupt the activity well before its models could reach external services.

The company also points to a live case in September. A model found a way to reach the internet from an environment meant to be isolated during a training run. Monitors flagged the breach within 15 minutes, and a responder began reviewing it three minutes later. OpenAI then paused frontier reinforcement learning while it strengthened the systems that contain its agents.

How the multilayer system works

OpenAI says it monitors every reinforcement learning (RL) run and evaluation for models at least as capable as GPT-6 Astra, as well as any harness that supports tool use. For most workloads, including RL and evaluations, it also monitors models it assesses as more capable than GPT-5.5 Thinking. All internal Codex traffic is monitored regardless of the model used. A version of the system also covers external tool use for models above the Critical threshold for cybersecurity capabilities under the company's Preparedness Framework.

To use compute efficiently, the system has several layers. Early stages spend little compute and are designed to pass cases along even when evidence is ambiguous. Later stages spend more on each case.

• Layer 0: lightweight classifiers called activation probes look for concerning patterns in a model's internal activations. They are tuned for high recall, accepting some false alarms. A random 5 percent of unflagged traffic is also sent to the next layer to catch what the probes miss. Frontier RL runs and internal Codex traffic skip this layer because of its sensitivity.
• Layer 1: a model-based reviewer examines the agent's recent conversation, chain of thought and tool use for signs of deception, unauthorized actions and attempts to manipulate an evaluation. For frontier RL runs and internal Codex traffic, OpenAI is moving to a schedule with no more than 30 minutes between checks.
• Layer 2: an investigative agent can check the purpose of an evaluation, read across the agent's context windows, and write and run code in an isolated environment. A security investigator looks for cases needing intervention, while an alignment investigator looks for behavior worth studying even without immediate security risk.
• Layer 3: for sensitive workloads, the security investigator compares transcripts with network logs, other system telemetry and evidence from earlier cases to decide whether to alert a human responder. Alignment reviewers compare reports across cases to find serious failures and recurring patterns.

Human response and its limits

When paging is enabled, the highest-priority security alerts reach members of the security and training teams, typically within 15 minutes of the initial alert. Responders review the evidence, examine affected systems and act to contain the incident.

OpenAI acknowledges that false alarms have made round-the-clock paging difficult for every workload. For frontier RL runs, a highest-priority alert can page a human at any hour. For most other traffic, paging is limited to business hours, and it is currently disabled for internal Codex traffic. Some critical alerts can automatically pause the affected workload without waiting for a responder. The company is working toward a broader safeguard that would pause affected activity if responders have not ruled out a highest-priority alert within 30 minutes of being paged.

During the Hugging Face incident, OpenAI says it quarantined the internal model prototype, encrypted its weights and restricted access to them, and paused all relevant workloads while adding safeguards. It is now exploring how to make such steps a standard part of its response to severe incidents.

What it means

OpenAI says the system demands substantial compute but that the cost is worth paying as models become more capable and the potential consequences of misalignment grow. It describes monitoring as one layer in a defense-in-depth strategy: alignment training aims to reduce harmful propensities, security controls limit what an agent can access or affect, and monitoring detects the failures that remain so the company can intervene and learn from them. The company states that all model developers should monitor the behavior of their most capable models.