Two new frontier models posted their best-ever scores on the Endor Labs Agent Security League | Blog | Endor Labs

What it is

An Endor Labs benchmark post (Agent Security League) comparing two frontier coding agents — Claude Fable 5.1 on Claude Code and GPT-6 Astra on Codex — on their ability to produce secure patches, not just functionally passing ones. Beyond the headline scores, the piece derives two decision-relevant conclusions about review policy and cost.

How it works

The benchmark distinguishes FuncPass (patch passes project functional tests) from SecPass (patch also passes the hidden upstream security test). The agent is never told a vulnerability exists. The key derived metric is the conditional rate: of patches that pass all functional tests, how many are secure. For Fable 5.1 that is 67/156 — 43% secure, so 57% of functionally-passing patches leave the flaw exploitable. Both new models converged near 42% conditional (up from ~30%), within 1.4 points of each other across two labs and harnesses — the post reads this convergence as a property of the problem, not a per-model weakness.

What to do

• Do not treat green CI as a merge signal for agent-generated security patches. The data says that is a coin flip. Because you cannot tell which of the functional passes are secure without checking, your review queue is every functional pass (all 156), not just the failures.
• Budget human review, not just compute. The post estimates that run generated ~$7,800 of engineer attention (156 patches × ½ hr × $100/hr) against $672 of compute — better than 10:1, and only the compute side is getting cheaper. Cheaper generation means more patches per sprint at the same per-item review cost.
• Cap wall-clock time as an economic control. Timeouts are billed in full and return nothing (Opus 5 timed out on 15 tasks, Astra on 29).

Cost mechanics worth knowing

Almost none of agent cost is code output — it is re-reading accumulated context once per tool call. Fable 5.1: 334M cache-read vs 6.24M output tokens (~53:1); Opus 5: 1,348M vs 9.37M (~144:1). Turn count is therefore the dominant cost variable: Opus 5 used 75 tool calls/task vs Fable 5.1's 42 (~1.6×) but burned 4× the cache reads, because each turn replays a context that has itself grown. "Wandering costs more than solving" — Fable 5.0 read more cache (437M vs 334M) for half the SecPass. Cost per verified secure fix: ~$10 (Fable 5.1) vs ~$19 (Opus 5).

Caveats

Fable 5.0's cache figure is a lower bound — 16 aborted runs went unrecorded. The conditional-rate convergence is inferred from two data points. All figures are Endor Labs' own benchmark, not independently reproduced.