Training State of the Art Vulnerability Discovery Agents through Reinforcement Learning | depthfirst

What it claims

dfs-mini1 is a vulnerability-detection agent from depthfirst. It is post-trained with reinforcement learning on gpt-oss-120b, together with a shell-only harness. On OpenAI's EVMBench Detect, which scores recall of high-severity smart contract vulnerabilities, it is reported as Pareto-optimal on cost versus performance. At pass@8 it exceeds every evaluated frontier model at 10-30x lower total cost. The page states these results only through figures. Its text gives no absolute pass rates and does not name the frontier baselines.

How it works

• Reward. The reward is continuous and based on overall detection coverage.
• Sandboxes. Thousands of Kubernetes sandboxes each hold one repository to audit. The agent's only tool is the container shell.
• Training data. Environments come from historical audits on multiple auditing platforms, with zero overlap with EVMBench. They span Solidity, Rust, Cairo and Vyper and average 50% more sLOC in scope. EVMBench, by contrast, is Code4rena-only and Solidity-only.
• Context. Context is capped at 32k, well below what gpt-oss-120b supports. The same policy is trained to summarize its own trajectory periodically, following ReSUM and SUPO. During training the agent learned to take more turns and to compact more often, and its summaries got shorter.
• Tools. The agent tried to install Slither and then fixated on its output, which was almost always false positives. depthfirst explicitly discouraged this during training. The team argues that fixed-methodology tools collapse the agent's strategy space and that a bare shell forces it to compose its own detection approaches.

Evidence

EVMBench Detect does not penalize false positives. Pass@k is therefore the union of detections across k rollouts and needs no oracle or vote. Pass@1, 2, 4 and 8 rise steadily with training steps. At pass@8, dfs-mini1 passes the frontier baselines, but those baselines were run at pass@1.

The cost advantage comes from two sources:

• Fewer input tokens. dfs-mini1 uses 2-3x fewer unique (cache-miss) input tokens per rollout than frontier models.
• Cheaper pricing. gpt-oss-120b is priced at $0.15/$0.60 per 1M input/output tokens.

RL raised reasoning-token use relative to the base model, but total output stays comparable to frontier models.

On Deep Vulnerable Hard, an internal benchmark built from historical CVEs across several languages and vulnerability classes, pass@1 and pass@3 improve "considerably" over the untrained base model. No numbers are given. That evaluation uses a harness, tools, prompts and codebases the model never saw in training.

How to use it

The page says nothing about releasing weights, a licence, an API or access.

Limits

• False positives. The number of vulnerabilities reported per task rose with training compute. Because the reward targets recall, false positives need downstream filtering.
• Grading favors volume. The headline pass@8 result comes from a grader that ignores false positives, so a union of 8 noisy rollouts is not penalized.
• Unequal comparison. The pass@8 result is compared against frontier models at pass@1.
• Internal benchmark. The CVE benchmark is internal and reported without figures in the text.
• Roadmap only. Precision-aware training and verification through end-to-end exploitation are stated as future work.