GPT-6.1 Sol on Codex: Astra-level security, a third faster, zero cheating | Blog | Endor Labs

What it is

Endor Labs' follow-up benchmark of OpenAI's GPT-6.1 Sol (released September 29, one week after GPT-6 Sol), run through the same Agent Security League Codex harness (Codex CLI 0.144.6) and the same 200 coding tasks used for GPT-6 Sol and GPT-6 Astra.

How it works

Head-to-head against Astra, GPT-6.1 Sol securely solves 6 tasks Astra misses while Astra secures 7 that GPT-6.1 Sol misses — of those 7, GPT-6.1 Sol still passes 5 functionally, so the residual gap is in security detail, not code comprehension. FuncPass gap is wider: 4.5 points (eight tasks), 77.7% vs Astra's 82.1%. Against GPT-6 Sol: 23 new secure solves gained versus only 7 lost, a ten-point FuncPass jump. GPT-6.1 Sol posts the best SecPass-to-FuncPass ratio of the four Codex models tested (43.9%) — when it produces working code, it's more likely than its siblings to have also made it secure. Cheating check: 13 flagged instances (patch similarity on 6, memorization on 5, edit-trajectory on 3, strict-test pass on 2, conversation analysis on 2), each adjudicated by two independent LLM rounds plus a tiebreaker — all 13 cleared, zero inspection errors, the third consecutive zero-cheating Codex run.

Preconditions / Affected

Applies to teams evaluating Codex-based coding agents on Azure at Sol-tier pricing ($2 input, $0.10 cached, $2.50 cache writes, $10 output per million tokens — cached input is half GPT-6 Sol's $0.20). Token/cost figures are estimates from raw counts since the Azure invoice wasn't yet finalized at publication.

What to do

• Treat GPT-6.1 Sol as a viable Astra substitute where cost matters: estimated $70–85 for the full 200-task run at list rates, versus $104 for GPT-6 Sol and $468 for Astra on the identical run (actual GPT-6 Sol/Astra figures were billed; GPT-6.1 Sol's is a token-based estimate pending invoice).
• Expect a third less wall-clock time: median 8 minutes/task vs ~11.5 for GPT-6 Sol and Astra; 60% of tasks finish within 10 minutes (vs 44–46%); only 9 timeouts vs 12–13; full four-worker run completed in 13 hours vs 19–20 hours.
• Token usage: ~235M input (94% cached reads) / 1.5M output, below both GPT-6 Sol (318M/2.05M) and Astra (276M/1.78M).
• If functional correctness matters more than the 4.5-point FuncPass gap, Astra remains ahead; if budget is the constraint, the SecPass gap versus Astra is a single task.

Caveats

Cost comparison is provisional — Endor Labs states the Azure invoice wasn't rated yet and they will update that section once final. The piece is explicitly framed as a short update to the prior GPT-6 Sol post, not a full methodology re-description.