nealbridges/VulnHunter

What it claims

The fork's main claim is about measurement. Upstream VulnHunter requires falsification and exploit tests but leaves open how hard to execute them: a static trace, a mocked test, or a real containerized server. Across six scans of one commit of a production Go service on five harness/model stacks over about three weeks, that discretion produced between 3 and 42 findings. It also produced opposite verdicts on the same sink: one run "proven" against a mock, another run closed it with a test against a real server.

The fork's planned fix is a Docker-first runtime-provisioning procedure, recorded per finding, plus PoCs whose impact is measured (rows leaked, ×-amplification, key-hours stranded) rather than described. This is marked "in progress" on the page and is not shipped, even though the summary lists it as a feature.

How it works

• Forward analysis. The scanner starts at attacker-reachable entry points (APIs, network messages, file uploads) and reasons forward. It does not start at dangerous sinks and search backward.
• Falsification pass. The same agent then tries to disprove each candidate finding. An independent adversarial verifier agent is roadmap only, so today the hunter grades its own work.
• PoC labelling. Findings without a working PoC are labelled as such.
• Portable skills. Each skill is a prompt file driven by the environment variables VULNHUNT_SKILLS_DIR, VULNHUNT_AGENTS_DIR, VULNHUNT_MODEL and VULNHUNT_HOST_CMD.
• Unattended operator. vulnhunter-run clones a target, hunts, locates the results, and writes and validates the manifest, with explicit stop rules.
• Benchmark tooling. Adds model configuration via environment variables, retry/backoff settings, and loss-point analysis for missed findings.

How to use it

Install. The script does not guess paths; set the directories for your harness:

git clone https://github.com/nealbridges/VulnHunter.git
cd VulnHunter
#   VULNHUNT_SKILLS_DIR="$GROK_HOME/skills" \
#   VULNHUNT_AGENTS_DIR="$GROK_HOME/agents" \
#   ./install.sh
./install.sh

• Windows: use .\install.cmd and .\uninstall.cmd.
• Files are copied, not symlinked, because symlinks break find/glob inside subagents. Re-run the installer after every pull.
• vh launcher: goes to VULNHUNT_BIN_DIR or ~/.local/bin, which must be on PATH.
• Phase files: if your harness doesn't follow skill subdirectories, add vulnhunt/phases/ to the session yourself.

Fixer. Needs git, an authenticated gh, and pip install -e ".[dev]" run in vulnhunter-fix/.

Verifier. Strictly read-only. Its only tools are Read/Write/Edit/Glob/Grep/subagent, with no shell and no network. You must pre-create out:

/vulnhunt-fix-verify repo=<abs_path> report=<abs_path> fixed=VULN-001,... out=<abs_path> [comments=<abs_path>] [additional_repos=<path1>,<path2>]

Batch and benchmark. Run these from harness/. Targets are listed in harness/local_harness/batch/REPO_LIST.txt.

python -m local_harness.batch.run scan --resume
python -m local_harness.benchmark.run --repos "name"
python -m local_harness.benchmark.run --tally-only

The benchmark pipeline is clone → scan → LLM judge → tally. Ground truth lives in ground_truth/<repo>.json, and engines are set in harness/local_harness/config.py.

vh (model-free).

vh write-manifest <results-dir> --exit-code 4
vh validate-manifest <results-dir>
VULNHUNT_HOST_CMD="<one-shot>" vh host <prompt-file>

Exit codes run 0–4, and an invalid value is recorded as 1. If you omit --exit-code, a non-empty README.md scan report counts as 0 and anything else as 1. write-manifest only writes a stub and never populates findings; vulnhunter-agent/ extracts the real findings.

Python 3.12+ is required, but only for the agent runtime and the harness.

Limits

• Sandboxed validation, measured-impact PoCs and the independent verifier are roadmap items. They are not current behaviour.
• Portability covers the skills only. vulnhunter-agent/ and harness/ are still Claude-Code-based.
• Benchmark evidence can't be checked from the repo. The 3–42 spread comes from one service at one commit. Raw artifacts are held by the maintainer and available only on request.
• The shipped ground truth is only a starting point. It is a minimal synthetic example mapped to NodeGoat, Juice Shop and WebGoat; you must build your own corpus.
• Commercial models may push back. Hosted models apply dual-use cyber safeguards, and aggressive exploitation can trip rate limits or usage flags.
• Development models are not the ones you'd likely use. Development used open-weight "abliterated/uncensored" community models, and no specific model is named.
• Scope: scan only code you are authorized to audit.
• Licence: Apache-2.0, inherited from Capital One.