Update on Security at METR

What it is

METR's post of 31 August 2026 is a disclosure of two security incidents at the AI evaluation nonprofit, written with its consultant Calif. It focuses on what the failures looked like in practice and which controls changed. METR says it believes no data in its two most sensitive categories (sensitive model access, and highly sensitive IP or business information) was accessed. It says the post is about outside attackers, not AI agents hacking during evaluations.

How it works

March 2026, stolen API key. A researcher with no sensitive access ran agents on a personal EC2 instance, deliberately public behind Google authentication. The vibe-coded app had a fail-open bug that silently disabled authentication, so it sat open to the internet for several days. METR suspects the attacker found it by scanning recently registered sites (for example certificate transparency lists) for LLM or agent keywords, to harvest model provider keys. The attacker told the agent to reveal its API key, added an SSH key for persistence, and used the key for three weeks. The usage was worth about $600,000 at list price, though the model developer had granted the credits free.

May 2026, probing campaign. METR was tipped off about financially motivated attackers who may have wanted frontier model access. They used agents to automate discovery, credential stuffing against authentication providers, OAuth token grant attempts, scanning of newly deployed services, and phishing of staff. Separately, the public transcript viewer exposed a read-only SQL query mechanism. It was scoped to public data by default, but a bug allowed access to unpublished evaluation data. That dataset was meant to hold only non-sensitive model data but included some sensitive model outputs. An independent researcher reported it, and METR took it offline and paid a bounty. The attackers touched the endpoint in passing, with no sign they found the exploit.

Preconditions / Affected

• Personal or ad hoc infrastructure holding organisational credentials, exposed publicly.
• Agent dashboards where auth can fail open.
• Keys with no spend ceiling, because credits were free and the provider offered no per-key limit at the time.
• Public apps sharing a data path with internal datasets.

What to do

The post lists these changes as METR's own, not as a general checklist:

• Ban METR credentials and data on non-METR infrastructure or devices.
• Require a security review before researchers deploy applications publicly.
• Add spend alerts to keys where possible, and monitor for unusual API key usage.
• Run public-facing apps in an isolated production environment, architecturally separate from internal infrastructure, so a public misconfiguration cannot expose internal data.
• Raise logging coverage on database queries and API usage, and cut noise alerts.
• Shorten credential lifetimes, narrow permission scopes, and shut down legacy infrastructure.
• Run regular threat modeling reviews and external red-teaming (Calif).
• Hire security staff: a security lead is in place and a full-time security engineer is planned.

Caveats

• The usage went unnoticed for three weeks for three stated reasons. Heavy token use and spurious rate-limit errors were normal for METR. The usage dashboard did not show rate-limited requests for all users. No spend ceiling existed on that key.
• During the May response, METR took the vulnerable endpoint down, then restored it after a few days without noticing the bug. The attackers did not retry it. METR calls it "highly unlikely" the attacker could have found and downloaded sensitive transcripts unnoticed, but some sensitive output data was accessible in principle.
• The attacker's discovery method in March is METR's suspicion, not a confirmed finding.
• The measures are stated as accurate as of 30 July 2026. METR's data tiers are described as simplified.