aws.amazon.comarticleAWS reports results for its autonomous pentesting agent, AWS Security Agent (now part of AWS Continuum), on an internal CTF-style benchmark of 37 recent 2025–2026 CVEs. Each CVE ran as a live docker-compose target with a hidden flag, and the agent started from an endpoint and a one-line hint with runtime CVE lookup disabled. It captured 35 flags, 33 (89.2%) through the intended vulnerability after an audit against canonical PoCs. The post describes the 7-stage pipeline (crawler, specialist agents, validator, exploitation agent, flag check, audit), three exploit walkthroughs (CMS upload-to-admin chain, AI workflow builder RCE, tRPC full-read SSRF), the two audit reclassifications caused by benchmark bugs, and two misses where the agent took the target down. It also argues that saturated public benchmarks like CVE-Bench and XBOW are contaminated.
AWS Security AgentAWSDocker+2
Opus 5CVEBenchSSRF+4
AI Security, Offensive Security, AppSec· Published and added 15 Oct 2026