Redwood Research Tests AI Monitors With Simulated Control Evaluations

The problem with testing monitors today

AI developers have started running monitors in agentic deployments. These are models that flag or block an agent's actions when they look dangerous. To judge whether a monitor is good enough, developers want to know how often it would stop a model that was actively trying to do something harmful, such as quietly inserting a security vulnerability, exfiltrating credentials, or sabotaging an experiment.

The standard measurement is a control evaluation: a model is instructed to attempt such attacks, and researchers record how often the monitor catches it before an attack succeeds. Developers typically run these in purpose-built control settings like BashArena or LinuxArena, where attacks can actually be executed.

Redwood Research argues that these settings offer only indirect evidence. Real deployments may involve particular codebases, services and tasks that the control settings do not represent. Running the evaluation in deployment itself would mean attacking your own systems, and building a faithful sandbox replica of every deployment context is impractical, according to the post.

How simulated control evaluations work

SCEs apply LLM environment simulation, already popular in alignment evaluations, to control evaluations. The process starts from transcripts of an agent working in a developer's actual codebase. The attacking model continues the task while pursuing a harmful objective, but its tool calls are never executed. Instead, an LLM simulator writes plausible tool outputs based on the transcript and the codebase, and an LLM judge decides whether the attack would have succeeded.

The obvious concern is simulator error. It might let attacks succeed that would fail in reality, or the reverse, produce transcripts that look different to the monitor, or change how the attacker behaves. To test this, the researchers took LinuxArena environments where attacks can be executed for real and ran the same control evaluations both in LinuxArena and in a simulated version.

They report that the gap between simulated and real results was often about as small as the gap between two independent real runs. They also found that results in simulated LinuxArena for a given control technique predicted measurements in real LinuxArena better than results in simpler control settings did.

Why not the alternatives

The post compares SCEs with two other ways of bringing control evaluations closer to deployment. The first is running on faithful executable copies of internal deployment. Deployments connect to the outside world through databases, storage buckets, Github, Slack and CI runners, and each dependency forces a choice: keep it connected to production and expose real systems, or replace it, which takes engineering and degrades fidelity. Redwood calls copies promising but says the exchange rate between realism and effort does not seem especially favorable.

The second is editing attacks into recorded transcripts, a method Redwood is currently developing. It produces strong attacks but cannot evaluate interactive protocols such as resampling or dynamic permissioning. It is also often unclear whether the model under evaluation could have carried out the edited-in attack. Redwood is optimistic about it as a complementary method. It says SCEs occupy a favorable tradeoff across environment realism, policy realism and cost.

Limits and what comes next

Redwood stresses that simulator fidelity can only be verified where real execution is possible, so applying SCEs to deployment involves some extrapolation. The group also does not expect SCEs to reveal the limits of a model's dangerous capabilities or to cover every kind of attack. The clearest gap is long-horizon, multi-agent attacks, for which no simulators have been built yet.

Because no records exist of an attack policy run in real deployment, the team used execution on LinuxArena as a stand-in for deployment and compared control metrics between executed and simulated runs. The post also describes a way to compare qualitative behavior, such as whether models carry out attacks in the same ways, by having a model summarize behavior along pre-specified dimensions of interest.

Redwood says the evidence is now strong enough that it would be excited to see AI developers running SCEs. For developers deploying monitored agents, the method offers a way to estimate monitor performance on their own codebases without executing real attacks.