https://github.com/dmdhrumilmistry/security-harness

What it is

security-harness also ships two concrete operational mechanisms the summary only gestures at: a cache/invalidation design for sh-pr-review, and a two-stage GitHub Actions pipeline (update-knowledge-base.yml → kb-review-and-merge.yml) that auto-merges KB updates.

How it works

sh-pr-review's merge-gate behavior rests on a tiering + caching scheme: Phase 1 reads fingerprints already posted on the PR before any hunter runs, so duplicate findings are never re-paid-for, only re-posted. A local sh-review-cache (OS cache dir, never the repo) hashes files, findings, and verdicts; the cache key includes every sh-kb-* knowledge base, model identity, skill version, and a 7-day TTL — any KB update invalidates all cached findings so a stale "clean" can't suppress a newly-detectable bug. A special case: if a base-branch merge changes the head SHA but touches none of the PR's own files or anything its findings depend on, the previous verdict is re-stamped onto the new SHA with zero agents launched.

The KB-freshness pipeline separates writing from shipping across two workflow files: stage 1 (update-knowledge-base.yml, scheduled every other day or manual) runs a generator agent that edits KB files from an allowlist (.github/kb-update/trusted-sources.md) with no shell and no push access, opens a PR on automated/kb-update, then dispatches stage 2 via workflow_dispatch — chosen specifically because a PR opened by GITHUB_TOKEN cannot trigger a normal pull_request event. Stage 2 runs an adversarial reviewer agent with no web and no shell that fails closed (any suspicion, uncertainty, or missing verdict file → REJECT) and only on APPROVE does a separate merge job run, which also refuses any PR that's closed, from a fork, or outside automated/*.

flowchart LR
    A[Generator agent edits KB] -->|opens PR, automated/kb-update| B[workflow_dispatch handoff]
    B --> C{Adversarial reviewer}
    C -->|REJECT: suspicious/uncertain/no verdict| D[blocked]
    C -->|APPROVE| E{Merge job checks}
    E -->|fork / closed / wrong branch prefix| D
    E -->|passes| F[Auto-merge to main]

Preconditions / Affected

Requires repo setting "Allow GitHub Actions to create and approve pull requests," and a credential secret (CLAUDE_CODE_OAUTH_TOKEN, ANTHROPIC_API_KEY, OPENAI_API_KEY, or GEMINI_API_KEY). For PR review, the running user needs write access to the target repo, checked before analysis so a 403 surfaces up front instead of after.

What to do

• Set branch protection on main requiring a PR and an approving review — this is honored by GitHub's own auto-merge path, so the KB-update pipeline queues for a human instead of landing unattended.
• For hardening, set KB_AGENT and KB_REVIEW_AGENT to two different model engines, so a prompt injection tuned for one is less likely to also fool the other.
• Inspect cache behavior via run.md, which records per phase what was launched, reused, or skipped — use --no-cache to disable or --refresh-cache to re-baseline if this drifts.

Caveats

The merge job falls back to an immediate (non-queued) merge only on repos where GitHub's native auto-merge is off, which weakens the human-in-the-loop guarantee unless branch protection is separately configured. The page states DAST runtime confirmation is still an unshipped, optional future item.