Skip to content

Back to the catalog

OpenAnt: LLM-Powered Vulnerability Discovery Through Code Decomposition, Adversarial Verification, and Dynamic Testing
arxiv.orgpaper

OpenAnt: LLM-Powered Vulnerability Discovery Through Code Decomposition, Adversarial Verification, and Dynamic Testing

OpenAnt is an open-source (Apache 2.0) vulnerability discovery system that combines static program analysis with LLM reasoning in a three-stage pipeline: decomposing codebases into reachability-filtered analysis units (cutting analysis surface up to 97%), running adversarial verification where the model simulates a constrained attacker to judge exploitability, and validating candidates via dynamic testing in disposable sandboxed containers. Evaluated against large real-world projects including OpenSSL, WordPress, and Flowise, it reportedly surfaced previously unknown vulnerabilities while cutting false positives and keeping analysis cost manageable, positioning closed-loop LLM+exploit-validation pipelines as a practical path to scalable automated security analysis.

TL;DR Open-source LLM pipeline (decomposition + adversarial verification + sandboxed dynamic testing) finds real unknown bugs in OpenSSL, WordPress, and Flowise with fewer false positives.
Open original ↗
Document

What it is

OpenAnt (github.com/knostic/OpenAnt, Apache 2.0, by Nahum Korda and Gadi Evron of knostic.ai). The summary calls it a "three-stage pipeline" — the paper actually specifies six stages, with distinct Claude models assigned per stage and concrete operational parameters the summary omits. Preprint dated June 2026.

How it works

The six stages, and where the summary's three collapse:

  • Stage 1 — Parsing. Language-specific AST extraction of every function plus a bidirectional call graph. Supported languages: Python, JavaScript/TypeScript, Go, C/C++, Ruby, PHP (Python/Go native AST, JS/TS via ts-morph, C/C++/Ruby/PHP via tree-sitter).
  • Stage 2 — Unit generation + reachability filtering. Functions become self-contained units with dependencies inlined transitively to a default depth of 3 levels. Entry points found via decorator analysis (@app.route, @router.get, @Get()), parameter patterns (request.args, req.query), and framework conventions; BFS from entry points marks reachable code. This is the 97% cut — OpenSSL: 15,232 functions → 390 units (97%); Grafana: 18,500 → 994 (94.6%).
  • Stage 3 — Exposure classification (Claude Sonnet, tool-assisted navigation). Four labels: exploitable / vulnerable-internal / security-control / neutral. Only "exploitable" proceeds. OpenSSL: 390 → 49 (further 87% cut).
  • Stage 4 — Vulnerability detection (Claude Opus, language-agnostic prompt). Five findings: vulnerable / bypassable / inconclusive / protected / safe.
  • Stage 5 — Adversarial verification. Any Stage-4 finding — including "safe"/"protected" — is re-tested. Attacker constrained to browser-only, no server access, no admin creds, no file modification. Requires multiple exploitation approaches before ruling a vuln absent, and a victim requirement: harm must affect a party other than the attacker (excludes self-impact).
  • Stage 6 — Dynamic verification (Claude Sonnet). Generates Dockerfile + test script + requirements (+ optional Compose for SSRF-style multi-service). Runs in a container with read-only filesystem, 512 MB memory, 1 CPU, no privilege escalation, 120s timeout. Output is a single JSON verdict: CONFIRMED, NOT_REPRODUCED, BLOCKED, INCONCLUSIVE, or ERROR. On ERROR, the error is fed back to the LLM and execution retries up to 3 iterations. All artifacts discarded after use.
flowchart TD
  A[Parse + call graph] --> B[Reachability filter ~97%]
  B --> C[Stage 3 Sonnet: exposure class]
  C -->|exploitable only| D[Stage 4 Opus: detect]
  C -->|else| X[dropped]
  D --> E[Stage 5: adversarial sim]
  E -->|victim requirement,<br>multi-approach| F[Stage 6 Sonnet: Docker exploit]
  F -->|ERROR| F2[feedback retry x3]
  F2 --> F
  F --> G[CONFIRMED / NOT_REPRODUCED / ...]

The evaluation argument (the paper's methodological core)

The summary omits why they test on OpenSSL/WordPress/Flowise rather than benchmarks. The paper argues standard suites (Juliet, OWASP Benchmark, CVE datasets) are invalid for LLM evaluation because they likely appear in training data — SE benchmarks show pass rates up to 4.9× higher on leaked vs non-leaked samples. It further cites vulnerability datasets with 20–71% inaccurate labels and 17–99% duplicated samples, and that LLM detectors score 10.5% higher on synthetic (Juliet) than real-world data. Juliet filenames (CWE89_SQL_Injection__*) leak the answer, enabling shortcut learning. Hence: evaluate only by discovering unknown vulns in live projects.

Context stat worth noting: the paper cites that only ~20% of developers actively use SAST, with false-positive rates ranging from single digits to over 40%.

Caveats

The fetched text cuts off in the middle of the real-world results section — the actual discovered vulnerabilities, counts, and per-project outcomes are not present in this excerpt, so the "previously unknown vulnerabilities" claim is stated but not enumerated here. Running the pipeline requires Claude API access (both Opus and Sonnet) plus Docker; cost scales with reachable-unit count.

Technologies named here

Techniques named here

  • Code decomposition by reachability filtering
  • Adversarial verification via constrained attacker simulation
  • Dynamic verification via automated exploit generation

Related in AI Security