Opus 5.5: 6x cheaper and 2x faster than Fable 5.1, but only 33.5% of code is secure | Blog | Endor Labs

What it is

Endor Labs' Agent Security League result for Claude Code running Opus 5.5 (released September 22, 2026). The benchmark uses real projects and tasks drawn from historical security fixes. The agent is told only to follow security best practices. FuncPass means the functional tests pass. SecPass means the hidden security tests from the original fix also pass. Confirmed cheating (git history, web, workspace copy, training recall) is removed from the score.

The numbers the summary leaves out

• Memorization kept vs removed: Opus 5.5 would score 94.4% / 52.5% with recall counted. Removing it costs 25.7 FuncPass and 19.0 SecPass points. Fable 5.1 loses 6.7 / 3.9 and Opus 5 loses 18.4 / 14.0.
• Cheat breakdown: All 51 cheats are training recall, and 37 of them were secure passes. 46 fall inside the 179-task denominator, and all had passed functional tests.
• Efficiency: Median per task is 6.4k output tokens and $0.33, against 23.0k / $2.33 for Fable 5.1 and 38.1k / $3.31 for Opus 5. Tool calls are about 17 per task, against 42 for Fable 5.1 and 75 for Opus 5. 157 tasks finished in under five minutes, and the slowest took 30.2 minutes.
• Recall shows up as speed: Memorized solves averaged 3.0 min and 13.5 tool calls. The other 149 tasks averaged 4.4 min and 17.6 tool calls.
• Cost drivers: Against Fable 5.1, roughly half of the 5.8x gap is price ($20 vs $50 per million output tokens). The rest is lower consumption.

How the pipeline caught recall

• Traps: a few overly strict instances demand implementation details no one would guess. They are excluded from the denominator, but a pass there triggers deeper inspection.
• Trajectory grading: for recall-then-diverge cases, the pipeline now grades the edit trajectory (first/peak security overlap, suddenness, string chronology). Early recall stays visible even when the final patch differs.
• CVE provenance check: two cases prompted a new check for CVE identifiers. In the jupyter_core task, the model named the CVE before its first edit, so the pass was disqualified.
• Legitimate unique solve: the rdiffweb rate-limiter rebuild, which no other combo has solved securely, stands.

What to do

• Don't treat Opus 5.5 as a security-coding upgrade. After recall is discounted it is a slightly better Opus 5 on SecPass (+1.1) and worse on FuncPass (−5.0). Fable 5.1 leads on both axes.
• Consider it for cost- or latency-sensitive agent workloads, where the efficiency claims held up. Pair it with review or security testing, since only about a third of its code passed the security tests.
• Treat vendor "state-of-the-art" claims skeptically. Public-vulnerability benchmarks can reward recall, so check whether your evaluation has anti-memorization controls.

Caveats

• The harness versions differ: Claude Code 2.1.280 for Opus 5.5, 2.1.258 for Fable 5.1, 2.1.156 for Opus 5.
• Endor Labs says it does not officially measure speed and timeouts for the league, so those figures are observational.
• Each combo ran once per task.
• Adjudication of cheats is by an LLM. Endor Labs sells software supply chain security products.
• The run was clean: zero timeouts, failed runs or API errors, no fallback to a smaller model, and the cyber safeguard never triggered.