Frontier models found the vulnerabilities. Only the attacker found the chains.

What it is

TaintedPort is Snyk's own deliberately vulnerable web application (v1.35, public at taintedport.com), built as a fixed benchmark with a known answer key: 57 vulnerabilities (34 commodity, 23 business-logic) and 15 registered exploit chains, each scored on a severity-weighted scale (Low 1, Medium 3, High 9, Critical 27). Total available points: 938 (587 from individual vulnerabilities, 351 from chains).

How it works

Each tool ran once (no medians). Evo COS ran gray-box: live URL plus source code, upgrading it from black-box to gray-box but never code-only. Claude Security ran white-box: source code only, using Mythos with extended thinking. The scoring credits a chain only when the tool demonstrates that its member findings actually combine at runtime — not merely that each member finding was individually reported.

The detailed example given: CHAIN-004. Every tool in the test found both member findings individually — an SSRF flaw and a hardcoded JWT signing secret. Only Evo COS proved the chain: it used the SSRF to reach the running app and retrieve the signing secret, then used that secret to mint a valid admin token, achieving full admin account takeover with a runnable proof of concept.

flowchart LR
    A[SSRF flaw] --> C{Found by both tools individually}
    B[Hardcoded JWT secret] --> C
    C --> D[Evo COS: retrieve secret via SSRF at runtime]
    D --> E[Forge admin JWT]
    E --> F[Admin account takeover - proven]
    C -.not proven.-> G[Claude Security: reported separately, chain not demonstrated]

Preconditions / Affected

Applies to tools being evaluated against a live, running target where exploit chains require runtime reachability (e.g., SSRF reaching an internal secret store, session/logout token survival, TOTP secret read nullifying 2FA). Business-logic chains especially — broken object-level authorization on profile updates, privilege escalation via forged JWT claims — depend on confirming each step is actually reachable in the running app, which source-only review cannot establish.

What to do

Numeric results: Evo COS confirmed 10 of 15 exploit chains; Claude Security confirmed none of the chains (it found individual flaws but could not prove reachability/chaining). Claude Security found one more critical-severity vulnerability than Evo COS (10 vs 9). Evo COS had higher precision (96.2% vs 90.2%) with fewer false positives and found the most total vulnerabilities overall. Findings reported only by Evo COS were overwhelmingly runtime behaviors (reflected/misconfigured responses, missing transport protections, tokens surviving logout, weak session handling), while findings Claude Security caught that Evo COS missed were mostly source-visible logic and cryptographic flaws that don't always surface through the running app. The actionable takeaway stated on the page: treat static code review (SAST-style) and live exploitation testing (DAST-style) as complementary layers, not substitutes — run both against the same target rather than relying on either alone.

Caveats

Single run per tool, not an average — results are one-shot outputs. Snyk built and maintains TaintedPort itself and states Evo COS was not tuned against it, but Snyk is also the vendor of Evo COS, so the benchmark is self-administered and not independently reproduced.