METR and Redwood stress-test OpenAI's internal agent monitoring, find gaps

A pilot with deep internal access

From July 21st to August 11th, the two researchers ran a pilot high-access exercise to stress-test OpenAI's internal misalignment monitoring system. The output was a 61-page internal report shared with OpenAI researchers and security staff. It focused mainly on estimating the fraction of agentic inference that is monitored, and on whether agents could in principle disable or subvert the monitoring system.

The access was extensive. The pair could view OpenAI's internal codebase, misalignment-monitor prompts and configuration details, and internal documentation. They also held technical interviews with approximately a dozen researchers and engineers across the monitoring, safety-systems and security teams. They worked on OpenAI-issued devices that never left the premises, and were accompanied while onsite.

METR says the exercise was designed with Redwood Research and OpenAI as a pilot for giving external researchers deep access to internal infrastructure, not as a source of robust public transparency. The parties agreed upfront that only a relatively brief, high-level post would be published. The assessment covers the system as it existed on and before July 21st, and OpenAI was expanding its monitoring during the exercise.

What the researchers found

On coverage, METR estimated that roughly 75 to 94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring, though it reports substantial uncertainty in both directions. For the subset used by internal Codex agents, the estimate was 99.2%. RL training runs were not covered at the start of the exercise, although OpenAI told the researchers a subset of rollouts had been monitored between March and June.

Unmonitored compaction events likely left on the order of 10% of eligible internal Codex traffic unmonitored between May and July 2026, METR says. The monitor triggered when agents returned control to the user, so agents could indefinitely postpone monitoring by continuing to act. OpenAI staff had spotted this beforehand and fixed it during the exercise.

Other findings included:

• A way Codex agents could in principle bypass Codex sandboxing and Auto-Review, allowing any action the host user could take.
• An exploit that would stop hundreds of recent actions from being monitored, with details redacted by OpenAI.
• Likely ways to craft inputs that make the monitor fail by exploiting serialization differences between what agents and monitors see.
• Two client-controllable headers that disable monitoring for new agents, though using them could itself be caught.
• Two unmonitored internal agentic tools thought to be read-only, for which a simple exploit could allow arbitrary actions in their sandbox. METR did not validate whether this was exploitable in practice.

There were reassuring results too. The monitor errored on less than 0.2% of requests according to information METR was shown, and one 24-hour period had zero load-sheds. METR verified that, with a few minor exceptions, the monitor sees the full trajectory the agent sees, and that its prompts look for scheming, monitor system subversion, sabotage, data exfiltration, deception and prompt injection, among other categories.

Design ideals and what comes next

Based on the findings, METR outlined design ideals, arguing that future monitoring infrastructure should withstand deliberate, strategic and sustained attacks from highly capable internal agents at scale. Its recommendations include:

• Make monitoring hard to avoid, for example by requiring explicit approval for unmonitored requests and monitoring agents every N messages or minutes rather than at the end of a turn.
• Keep careful accounting of all inference and cross-reference it with monitor logs.
• Respond quickly to incidents, with clear ownership and tamper-proof, searchable logs.
• Red-team the monitoring system actively, including control evaluations and methods to assess strategic collusion.
• Isolate agents from their monitors, so that agent actions cannot influence whether or when monitoring runs.

OpenAI has also described significant subsequent changes to the intended coverage of its monitoring system. METR has not evaluated those claims, and the exercise did not establish the expanded system's current coverage. For future versions, METR says it expects to follow the transparency and independence practices in its document on working with AI companies, to provide more public detail. The post notes it follows a similar earlier stress-testing exercise with Anthropic, and that OpenAI had the right to redact the public version.