The state of AI for security: Measuring what matters most for building trust

What it is

AWS Security's Deception Benchmark, released alongside a whitepaper and GitHub repo, is built on two distinct challenge types: 6,988 "code-level" samples where vulnerable and safe variants differ by a subtle fix, and 2,707 "environment-gated" samples where identical code is exploitable or not depending on deployment context (e.g., a Kubernetes Network Policy or IAM boundary).

How it works

Samples were built through an adversarial loop — generate, test against frontier models, harden, repeat — discarding any sample a model got right easily, so the benchmark is calibrated at the frontier rather than below it. Generation and hardening consumed tens of billions of tokens. Labels go through a convergent audit loop: multiple blind independent reviewers, escalation to adjudication on disagreement, and unresolved disputes are moved to the unscored pool rather than relabeled — so a bad challenge can be dropped but a wrong label can never enter the scored set. The team reports convergence under a 3% dispute threshold, targeting under 1% surviving human adjudication, and a 100-sample human audit found no label errors.

What to do

Two concrete mitigation levers the post names: proof-of-exploit prompting, and agentic/multi-step verification harnesses. Proof-of-exploit prompting cut false positives by 17–74 points but increased false negatives, missing 7–44% of real vulnerabilities — a direct tradeoff, not a fix. Multi-step agentic harnesses were explicitly not tested here; the post flags them as a different operating point and invites builders to run them against the benchmark to see if scaffolding closes the gap. For teams evaluating AI security vendors now: ask for performance on distinguishing real vs. false-positive vulnerabilities (not just detection rate), and pair AI review with human verification on high-risk code paths. The dataset and submission instructions for verified scoring are on GitHub.

Caveats

The post itself cautions that agentic verification is proven mostly on offensive tasks with a clear success oracle (exploit fires or not); judging "this code is safe" has no equivalent oracle, so extra iterations may just re-sample the same judgment rather than confirm a negative — meaning a harness still inherits the base model's single-turn understanding gap. Labels are withheld from the public release specifically to prevent the benchmark from becoming a memorization target, and roughly a third of samples (5,127 of 14,822) are unscored and mixed in for the same reason.