Red-Teaming OpenAI's Internal Agent Monitoring Systems

What it claims

METR and Redwood Research ran a three-week pilot (21 July to 11 August 2026) to stress-test OpenAI's asynchronous internal misalignment monitor. One employee from each organisation took part, with access to OpenAI's codebase, monitor prompts and configuration, and internal documentation. They also interviewed about a dozen engineers across the monitoring, safety-systems and security teams. The private output was a 61-page report. The public post lists the findings OpenAI allowed to be published, plus five design ideals for monitors that must withstand "deliberate, strategic, and sustained attacks from highly capable internal agents at scale."

Evidence

The coverage figures apply to the system as it stood on or before 21 July 2026:

• Responses API coverage. An estimated 75 to 94% of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring. For the internal Codex subset, the figure was 99.2%.
• Eligible traffic that was dropped. "Eligible" means the traffic is intended for the monitor. Load shedding, monitor errors or bugs can still drop it. The monitor errored on under 0.2% of requests, and one 24-hour period had zero load-sheds.
• Compaction events. Compaction events were unmonitored, which likely left on the order of 10% of eligible Codex traffic unmonitored from May to July 2026. OpenAI had found this independently and fixed it during the exercise.
• Monitoring trigger. The monitor ran only when an agent returned control to the user. While compaction was unmonitored, an agent could postpone monitoring indefinitely by continuing to act.
• RL training. RL training runs were not covered at the start of the exercise. Some rollouts had been monitored between March and June.

The exploitable gaps were:

• Codex agents could in principle bypass Codex sandboxing and Auto-Review, gaining the host user's full permissions.
• One redacted exploit would stop hundreds of recent actions, including its own, from being monitored.
• Serialization differences between the agent's view and the monitor's view could let agents craft inputs that make the monitor fail, so the trajectory is never escalated for human review.
• Two independent client-controllable headers disable monitoring for new agents. Setting them and spawning the new instance could itself be caught by the monitor.
• Two unmonitored internal tools assumed to be read-only had a simple exploit that allowed arbitrary actions in their sandbox. This was not validated in practice.

Some checks came out positive. Apart from minor exceptions, the monitor sees the full trajectory the agent sees. Its prompts cover scheming, monitor subversion, sabotage, data exfiltration, deception, and inbound and outbound prompt injection.

How to use it

The design ideals are the part of the post that carries over to other monitoring systems:

• Make monitoring hard to avoid. Unmonitored inference should need explicit approval for specific organisations or API keys that are barred from agentic use. Agents should be monitored every N messages or minutes, not at turn end.
• Account for coverage. Log all inference and cross-reference it with the monitor's own logs. Keep a list of known bypasses. Make sure the monitor sees the same context as the agent.
• Respond to incidents. Assign clear ownership and guidelines. Keep tamper-proof, searchable logs of inference and monitor traffic.
• Red-team the monitor. Run periodic control evaluations and bug searches, and develop methods to evaluate strategic collusion.
• Isolate agents from monitors. Run scaffolds apart from where tool calls execute. Agent actions should not affect whether or when the monitor runs. Agents should not normally see the monitor's design or code.

Limits

The findings describe the system as of 21 July 2026, not the expanded system OpenAI was building at the time. OpenAI says all production-impacting vulnerabilities are patched and that coverage has changed significantly. METR has not evaluated either claim.

The coverage estimates exclude other internal agentic traffic, such as agents on OpenAI's internal research inference service. METR could not measure that volume. The 75 to 94% range carries "substantial uncertainty in both directions." The compaction estimate is rough because compaction-triggered monitor calls had not been logged. METR could not check whether agents had actually exploited the postponement gap.

OpenAI could redact anything at its sole discretion, and METR had no right to describe those redactions. Leadership at METR and Redwood received a 38-page redacted version of the report. The exercise was a pilot of the access model, not a public transparency report.