subScraper

What it is

subScraper (Recon Command Center) run as an operator: the page documents how you actually launch and secure it, which the summary omits. It's driven from a single main.py that serves a web SPA and, by default, binds to all interfaces on http://0.0.0.0:8342 — an unauthenticated-by-config recon dashboard reachable on your network unless you lock it down.

How it works

First run triggers an interactive setup wizard (wordlist path, concurrent jobs, nikto prefs, and optional API keys for Amass/Subfinder). It writes tool config files and prints next steps. State persists in recon_data/state.json (resumable jobs) and recon_data/completed_jobs.json (history across restarts). It ships a login system with admin and regular roles, user CRUD in the UI, and can serve over HTTPS with an auto-generated self-signed cert or your own. Concurrency is one "workers per tool" knob (default 5); for httpx/nuclei/nikto it splits a single host batch across workers, so raising it speeds up one scan, not just parallel jobs.

Preconditions / Affected

Anyone standing this up — because the default bind is 0.0.0.0 and setup can be skipped, an unhardened instance exposes a scanning console (and the /api/agent/* control API) to the local network.

What to do

pip3 install -r requirements.txt
python3 main.py                    # first run launches the setup wizard

Serve it over TLS rather than plaintext:

python3 main.py --https                                   # auto self-signed cert
python3 main.py --https --cert /path/to/cert.pem --key /path/to/key.pem

One-off CLI target (bypasses the UI):

python3 main.py example.com --wordlist ./w.txt --skip-nikto
python3 main.py 'acme.*'          # wildcard TLD expansion from Settings
python3 main.py '*.apps.acme.com'

Docker (all tools pre-installed, amd64/arm64/armv7) via Compose is the recommended path if you don't want to solve tool installation. The agent API uses scoped, revocable API keys — provision one per program rather than sharing.

Caveats

--skip-setup is offered but flagged "Not Recommended"; skipping leaves tools unconfigured. HTTPS and user management details live in USER_MANAGEMENT_AND_HTTPS.md, cleanup/pagination in CLEANUP_AND_PAGINATION.md, and the agent API in AGENT_API.md — none of which the page reproduces.