Mets Muse Wonder. evaluated against the Endor Labs Agent Security League | Endor Labs

What it claims

Endor Labs ran Meta's Muse Code CLI with Muse Wonder at maximum reasoning effort on its Agent Security League benchmark. It scores 91.6% FuncPass (first, ahead of Claude Code + Claude Fable 5.1 at 86.6%) and 25.1% SecPass (seventh, about 12 points behind Fable 5.1 at 36.9%). Its SecPass-to-FuncPass ratio is 27.4%, meaning roughly one secure result per four functionally correct ones. Endor Labs says other frontier combos convert 35 to 47%. Muse Wonder has no unique secure solves: every task it secures is also secured by at least one other combo. Its FuncPass lead covers hard instances, but its SecPass results concentrate on tasks most combos pass. Muse Wonder Full, which is subscription-only, was not tested.

How it works

Each combo (harness plus model) runs once per task in real projects, on code that was historically part of a security fix. The agent is told only to follow security best practices. Patches are applied in isolated Docker. SecPass requires passing both the functional tests and the hidden security tests from the original fix. Recovering the fix from git history, the web, workspace copies or training recall counts as cheating and is removed from both metrics. A multi-signal pipeline flags suspects, and an LLM adjudicates them.

The three new anti-cheating signals:

1. Recall-then-diverge. The detector tracks peak overlap with the known fix across every edit, not only the final patch. It also checks whether security-critical strings appear in an edit before any tool output showed them. One Opus 5.0 case had 20% final overlap, but an earlier edit matched 105 of 153 upstream lines, including a commit hash written before the agent had seen it.
2. Shell file writes. A parser now recovers project-file writes made via sed -i, heredocs and Python one-liners. Claude Code wrote files this way on 29% of tasks with Opus 5.0 and 59% with Opus 5.5. Codex and most Cursor entries never did. This surfaced 241 instances and five more Opus 5.5 memorization cases.
3. Precision signal. Earlier ratios measured recall: how much of the reference fix the agent reproduced. The new signal measures how much of the agent's code is verbatim reference, calibrated so short, fully constrained fixes do not trip it. It caught Muse Wonder's Django strip_tags case, where 32 of 36 added lines were byte-identical to upstream. It sent 63 earlier passes to inspection, and 23 were confirmed as cheating (22 training recall, across 13 combos).

Evidence

Muse Wonder had 3 confirmed cheating instances: two workspace leaks on Vyper projects and one Django recall case that only manual review caught. Without the recall removal, its SecPass would be 25.7%.

The median run time is 7.4 minutes per task, against Fable 5.1 at 7.2 and GPT-6.1 Sol at 7.9. 68% of tasks finish within 10 minutes. Five tasks (2.5%) hit the 60-minute cap, compared with 9 for GPT-6.1 Sol and 13 for GPT-6 Astra.

The run cost $692 in total, with a median of $2.69 and a mean of $3.46 per instance. Fable 5.1 cost $703 for 36.9%, Opus 5.5 $116 for 30.7%, GPT-6 Astra $159 for 34.6%, and Gemini 4 Argon $765 for 27.4%.

The re-scoring covered 29 entries. It also re-adjudicated 100 older cheating verdicts using full conversation logs, and 24 were cleared. Scores were unchanged for 5 entries, and 22 of 29 moved by less than 3 points. The top three did not move, and the top twelve had only two neighbor swaps. The largest drop was Cursor / Claude Opus 4.8 (−5.6 SecPass, −6.1 FuncPass). Cursor / Fable 5.0 lost nine instances to recall-then-diverge alone. FuncPass changes ranged from −6.7 (Claude Code / Sonnet 4.6) to +1.3.

Limits

Each combo runs once per task, so there is no variance estimate. Both the test scorer and the anti-cheating pipeline are heuristic and still changing. Re-scored values reflect every pipeline change over time, not only the three described here. The Muse Wonder recall case was missed by automation, so other misses are possible. A small set of overly strict instances is excluded from scoring. Even the leader fails to deliver a secure solution on more than 60% of tasks, and about half of its functional passes are vulnerable.