Endor Labs: Claude Fable 5

What it is

An Endor Labs "Agent Security League" benchmark follow-up pairing Claude Fable 5 with the Cursor agent on the same 200 real-world vulnerability-fixing tasks. Cursor + Fable 5 reaches the top fair score to date: 72.6% FuncPass, 29% SecPass, versus 59.8%/19.0% for Fable 5 under Claude Code — a +12.8pp / +10pp swing from harness alone.

How it works

Each "combo" (harness × frontier model) gets a coding task in a real project and is told only to follow security best practices — never that the missing code is security-critical. Patches run in isolated Docker. FuncPass = passes functional tests available during development; SecPass = also passes hidden security tests from the original CVE fix (so it must first be functionally correct). Endor attributes Cursor's edge to patch quality and its tendency to steer the model toward the security dimension — not extra time or infrastructure.

The anti-cheating pipeline matters for reading the numbers. It suppresses git-history and web retrieval via prompt instructions, then runs post-hoc trajectory checks and patch-vs-known-fix similarity with LLM adjudication. Memorization (training recall) is the hard case, flagged not by generic similarity but by workspace-underivable artifacts: verbatim upstream comments, changelog annotations, and CVE/CWE identifiers appearing nowhere in the task or codebase. Overly strict tests (e.g. exception strings copied from the reference patch) are used as traps — passing them is itself a cheating signal. Confirmed cheats are removed and unfeasible instances excluded from the denominator to produce "fair" scores.

What to do

This is decision input, not a patch. If you are relying on an AI agent to fix vulnerabilities:

• Treat the harness as a security-relevant choice, comparable to or larger than model choice.
• Do not assume a functionally-correct AI patch closes the vulnerability — even the best combo leaves roughly 7 of 10 working patches with the vuln still open.
• If comparing your own results, note the leaderboard context: prior front-runners were Codex + GPT-5.5 at 22.3% and Cursor + GPT-5.5 at 24.0% SecPass.

Caveats

• Cheating remained high even for the winner: 29 confirmed instances, 28 of them training recall. The 29% SecPass is after removing these; raw performance is inflated by memorization the benchmark specifically excludes.
• Fair-score denominators drop instances, so figures are not directly comparable to naive pass rates.
• Cursor + Fable 5 solved five security instances no other combo had cracked — a capability signal, but still within a sub-30% ceiling.
• All results are one run per task; no variance reported.