Codex with GPT-6 Astra posts 82.1% FuncPass and 34.1% SecPass | Blog | Endor Labs

What it is

Endor Labs' benchmark of OpenAI GPT-6 Astra running in Codex on their Agent Security League suite. The summary already gives the headline scores; what it omits is the operational trade-off that determines whether Astra is worth running: speed, token cost, and timeout behavior.

How it works

The benchmark runs each combo (harness + model) once per task on real projects containing a historical security fix. FuncPass = passes functional tests; SecPass = also passes the hidden security tests, so a secure result must first be functionally correct. A multi-signal pipeline flags suspected recall (git history, web, workspace copy, training recall), and an LLM adjudicates each flag; confirmed cheating is removed. This run added trajectory grading — first/peak security overlap, suddenness, string chronology — to catch "recall-then-diverge," where an agent recalls the known fix early but the final patch has diverged. A small set of "overly strict" instances is excluded from the denominator but kept as traps: a pass triggers inspection rather than automatic credit.

The cost of Astra's score

This is the part the summary doesn't carry — the score comes at a real time/token price:

• ~16.8 min/task and 29 timeouts for Astra, versus ~9.5 min/task with zero timeouts for Claude Code + Fable 5.1, which posts a higher score (87.2%/36.9%) in roughly half the wall-clock.
• Versus GPT-5.6 Sol, Astra spends ~1.8× wall-clock and ~1.6× tokens to buy the +14.5-point jump on both axes. The extra tokens are input, not output (more searching, not more writing).
• Astra issues a similar command count to Sol but searches more, edits less, and leaves fewer leftover shell calls — the gain is conversion, not activity.
• Timed-out runs still ship a patch: Codex submits whatever diff exists at the kill boundary, so a capped run isn't a zero.

Caveats

Note the SecPass figure is quoted inconsistently on the page (34.1% in the results table, 34.6% in the intro/takeaways). The cheating comparison is central to reading the board: Fable 5.1's lead survives a 17-cheat discount and Opus 5's raw SecPass fell after 38 confirmed cheats, while Astra had zero to remove — so its numbers are directly comparable without a memorization haircut.