How an assessor could investigate control and monitoring of internal AI agents

What it claims

METR's October 2026 proposal argues that LLM-based monitoring is the main evidence frontier developers have about agent misbehavior. Sandboxing and least-privilege permissions do not cover agents with broad affordances such as internet access. A control assessment should therefore test whether monitors run, whether they can be broken or fooled, and whether flagged incidents reach a human who acts. METR says the reasoning reflects how it evaluated its own internal monitoring system. It will revise the sub-questions as new capabilities and misaligned behaviors emerge.

How it works

Coverage. The assessor cross-references inference logs against monitor logs to find unmonitored paths. Employee interviews catch inference sources that the logs miss. The assessor also checks whether eligible trajectories are silently dropped, whether multimodal data reaches the monitor, and whether blocking monitors guard safety-critical areas. Unmonitored inference must be shown to be non-agentic. A compromised agent could co-opt "limited tool use" paths, so the label alone is not enough. Git history helps estimate how long a coverage gap existed.

System vulnerabilities. The assessor reads the critical-path code: the inference and monitor trigger system, internal APIs and serving code, and agent scaffolds. It then fires the production monitor on known attacks and inspects the full input and output. METR says many bugs sit in trajectory pre- and post-processing, compaction and memory handling. Questions include whether agents can disable monitoring, alter trajectories before evaluation, spawn unmonitored agents, or reach monitors through internal supply-chain attacks.

Monitor evasion. There are three techniques:

1. Mine real transcripts for attack opportunities and build attacks from them.
2. Modify incidents the monitor flagged until they pass.
3. Search for severe incidents that were never flagged.

A large transcript pool matters for a second reason. Capable agents may pass up easy attacks and take only ones the developer would reliably miss. METR also recommends testing monitor–agent collusion with honeypots for monitors and by searching monitor chains-of-thought for discussion of collusion.

Response. The assessor measures escalation thresholds and time to human review. It tries to get flagged attacks past reviewers by making them resemble common false positives. Test trajectories are inserted into the live pipeline using standard penetration-testing precautions.

Impact. The assessor maps attack chains and the persistence agents could reach, preferably by testing on real devboxes, containers and inference systems.

How to use it

The access list goes beyond the summary:

• At least ~100k representative agent transcripts from employee opt-in usage, plus a similar volume from evals and training. Hundreds of thousands to millions is preferred.
• At least ~100 transcripts of the most severe internal incidents, including incidents employees reported through other channels.
• Codebase access comparable to the developer's frontier AI security researchers. Sensitive code off the critical path may be removed.
• Inference access to the most capable internal models.
• Limited runtime cluster access, because some configuration is not stored in code.

Logs need only be roughly representative. Teams doing compartmentalized work such as pretraining may be excluded if their agent setups do not differ significantly.

Limits

Coverage estimates assume the logs have not been tampered with. That assumption depends on the vulnerability and evasion work. Transcripts are too sensitive to approve one by one, so the assessor needs broad pre-approval to view them incidentally. Routing sampling through a company employee adds overhead and can reduce independence if instructions are miscommunicated. Replacing infrastructure access with staff interviews is a meaningful limitation. Stripping sensitive code can be hard when dependency chains are complex. The document is a proposal and reports no assessment results.