Summary of METR's predeployment evaluation of Claude Opus 5.5

What it claims

METR's summary evaluates how Claude Opus 5.5 affects AI R&D, structured around two questions: (A) would using Opus 5.5 now dramatically accelerate AI R&D at Anthropic, and (B) was Opus 5.5's own development already dramatically accelerated by AI. On both, METR concludes no. Opus 5.5 is judged a modest, on-trend improvement over Fable 5.1 — incremental across quantitative evals, not a discontinuous jump — unlikely to fully automate AI R&D. The report explicitly does not verify compliance with any Anthropic policy threshold and does not assess alignment properties.

How it works

The assessment ran over 10 business days of API access and combined five capability tasks with Anthropic-supplied questionnaire/interview material and prior trend data (METR's Frontier Risk Report). The five tasks:

• Budget NanoGPT Speedrun — constrained AI R&D speedrun (verifiable)
• Gaming Bot — write Python to play a game via non-visual API (verifiable)
• LMCA — conceptual argumentation dataset (harder-to-verify)
• Sunlight — open-ended research + report writing (harder-to-verify)
• Train a Program — train an ML model to replicate a piece of software's behavior

A key input was a separate METR team's preliminary internal AI R&D acceleration report; that team had elevated access and shared only conclusions, not evidence, so METR uses it as input without defending its claims.

Evidence

• Opus 5.5 improves over Fable 5.1 on both verifiable and harder-to-verify tasks, but incrementally.
• Anthropic states Opus 5.5 continues the "Mythos-level"/AECI trend on Anthropic ECI (from Mythos Preview onward).
• On (B), the separate internal report estimates ~1.5X overall capability acceleration due to AI (1.5 years in 1 year), with ~30% chance of 2X. METR flags that the report did not specify the time period, so it's unclear whether this even applies to Opus 5.5's development.
• METR argues full AI R&D automation needs large gains in "judgement"/"taste" (foresight, prediction, creating one's own feedback loops), where Opus 5.5 shows no large improvement.

Limits

• Self-described preliminary evaluation; conclusions are qualitative and hedged ("highly uncertain").
• The data cannot distinguish consistent, accelerating, or decelerating rates of R&D progress — incremental gains remain consistent with rapid overall progress.
• Independence caveat: conducted under an unpaid agreement; Anthropic reviewed and edited the text, and METR signed off on the version in the system card. A transparency provision lets METR disclose the redaction process, whether redaction rights were used, and whether findings depended on redacted material — but the summary text here does not state those outcomes.
• Two sources are undisclosed: the separate team's supporting evidence, and an additional source used to understand Anthropic's R&D (stated not to bear on this model specifically).