ThreatRadar

What it is

ThreatRadar is a self-hosted, Dockerized threat-intelligence pipeline (nine sequential microservices) aimed at SOC/CTI teams. The summary covers its intent; this page adds the concrete deployment shape, the detection ordering, and the operational constraints an engineer needs before standing it up.

How it works

Poisoning detection is two-staged, not parallel: the scikit-learn IsolationForest (calibrated contamination scoring) runs first and flags statistically unusual IOCs; only those flagged then go to the GPT-4o (via OpenRouter) semantic check against 8 contradiction classes. The LLM returns one of three verdicts — COHERENT / SUSPICIOUS / CONTRADICTED — before anything reaches analyst queues. This gating matters for cost: the LLM is not called on every IOC.

IOCs are normalized into seven typed Elasticsearch indices (IP, URL, file hash, domain, CVE, ransomware family, crypto wallet), each routed to type-appropriate analyzers. Cortex scoring produces composite scores with CRITICAL/HIGH/MEDIUM/LOW tiers and recommended actions. MISP push includes TLP tags and distribution controls with configurable worker concurrency; a daily refresh re-syncs verdicts and requeues IOCs whose MISP events went missing. Analyst sightings drive incremental retraining with configurable signal thresholds and cooldown periods.

flowchart LR
  A[Ingest feeds] --> B[Normalize into 7 ES indices]
  B --> C[Enrich: MITRE/CVSS/actor]
  C --> D[Cortex scoring]
  D --> E{IsolationForest<br/>anomalous?}
  E -->|no| G[Push to MISP<br/>+TLP tags]
  E -->|yes| F{LLM: 8 classes<br/>COHERENT/SUSPICIOUS/CONTRADICTED}
  F -->|coherent| G
  F -->|contradicted| H[Held from queue]
  G --> I[MISP sightings] --> J[Retrain model]
  J -.feedback.-> E

Preconditions / Affected

Running this requires:

• Python 3.11.x or 3.12.x
• Docker 24.x or later, docker compose v2.x
• Windows: WSL2 only — native Windows is not supported.

External API keys are needed for the six Cortex analyzers (VirusTotal, AbuseIPDB, Maltiverse, IPinfo, Urlscan, HybridAnalysis) and OpenRouter/GPT-4o. All secrets are env-var injected; .gitignore excludes .env by default.

What to do

python3 --version    # 3.11.x or 3.12.x required
docker --version     # 24.x or later
docker compose version  # v2.x required

git clone https://github.com/THREATRADAR-Pipeline/ThreatRadar.git
cd ThreatRadar

For running scripts outside Docker (local dev):

python3 -m venv venv
source venv/bin/activate
pip install -r elasticsearch/requirements.txt
pip install -r elasticsearch/poisoning_detection/requirements.txt
pip install -r misp/requirements.txt

The page explicitly warns: verify .gitignore excludes .env before pushing.

Caveats

Retention runs on a 180-day rolling window in Elasticsearch — transient IOCs expire automatically, though CVEs, ransomware families, and crypto wallets are kept as permanent reference data. Plan for that expiry if you expect long-lived history for other IOC types.