Agent Threat Rules (ATR) — open detection-rule standard for AI agent threats

What it is

Agent Threat Rules (ATR) at Working Draft 3.0.0-alpha.1, evaluated by the TypeScript reference engine (agent-threat-rules on npm) and pyatr. Beyond the summary, the page documents a two-switch runtime model with deployment traps sharp enough to be the reason an engineer opens it — and real, corrected benchmark numbers.

The runtime model that isn't in the summary

Two independent switches govern atr guard, and neither implies the other:

• Lane (enforce / alert / hunt) selects which rules may fire. Default is hunt — every rule, report-only.
• Blocking decides whether ATR may act. Off by default; turning it on is the explicit operator directive SPEC.md §5.5 requires before automatic response actions.

ATR never emits permissionDecision: "allow" in either mode — in the Claude Code hook contract that is an affirmative approval suppressing the host's own prompt. It emits a decision only to restrain (deny/ask), and only with blocking on.

The deployment traps

--lane enforce costs recall, and the cost is large. Enforce loads only maturity: stable rules — 106 of 777 live rules at the documented commit. Narrower firing set, fewer attacks caught. Re-derive before quoting (eight rules quote maturity: "stable", so grep is wrong):

python3 - <<'PY'
import glob, yaml
rules = [yaml.safe_load(open(p, encoding='utf-8'))
         for p in glob.glob('rules/**/*.yaml', recursive=True)]
live = [r for r in rules
        if r.get('status') not in ('draft', 'deprecated')
        and str(r.get('maturity') or '').strip() != 'deprecated']
stable = [r for r in live if str(r.get('maturity') or '').strip() == 'stable']
print(f'files={len(rules)} live={len(live)} enforce={len(stable)}')
PY

SKILL.md scanning is 0% in enforce. All 38 scan_target: skill rules are maturity: test; none is stable. So the "100% SKILL.md recall" figure is a hunt-lane number — in enforce, 0 of 32 malicious samples fire. Deploying enforce-mode expecting skill coverage forms a completely wrong expectation.

Config footguns:

• new HookHandler({ blocking: "false" }) — a non-empty string is truthy, so "false" switched enforcement on while reading as "off".
• An unrecognized ATR_LANE warns on stderr and falls back to the broadest lane with blocking forced off (exit 0), because a non-zero exit in a Claude Code command hook discards every detection silently.
• Library constructors (ATREngine, ActionExecutor, HookHandler) ignore ATR_LANE/ATR_BLOCKING — an embedded engine can't be re-pointed by environment.

Benchmark numbers the summary only gestures at

The "self-critical" framing has concrete figures worth citing:

• garak (NVIDIA in-the-wild corpus): 98.0% (3.0.0) → 92.3% (3.5.12, 784 rules). The 95.7%→92.5% step is a real recall loss from PR #327 tightening an over-broad persona regex in ATR-2026-00001, not a counting change.
• hh-rlhf / ATLAS / OWASP LLM fell 99.1%→1.5%, 100%→39.0%, 100%→16.1% once the runner was found to never have called the engine — the old shadow matcher miscompiled [\u{E0001}\u{E007F}] (no u flag) so it matched any text containing "e", i.e. it measured "how many samples contain a vowel."
• Corpus overfitting is disclosed: PromptInject 100% is 7 of 780 rules; 5 were written from that corpus — remove them and recall is 9.7%. PromptBench 15.7% is 3 rules, two mined from it (2.4% without).
• The prior lane FP rates (enforce ~0.24% / hunt ~9%) and garak 91.5%/56.9% are withdrawn — no backing measurement file; CI now fails on uncited numbers via scripts/check-benchmark-citations.ts.

Ecosystem / provenance

CVE-driven response is real: 2026-05-07 MSRC Semantic Kernel CVEs (CVE-2026-26030 lambda+eval RCE, CVE-2026-25592 autostart file write) → Copilot opened a regression-fixture PR, ATR v2.1.2 rules merged/published 2h 16m later — though coverage was partial (2 of 4 fixtures) and this is Copilot-in-AGT, not an MSRC endorsement. Adopters self-declare via PR; standardization scaffolding (9-seat TSC, ATR-Certified™, OASIS submission) is all tagged PROPOSED, not ratified — single-maintainer BDFL today.

Caveats

Cloud reporting is fully opt-in: nothing leaves the machine without --report-to-cloud, which also requires --tc-url/ATR_TC_URL (no default recipient). Sent data is a contentHash + rule id + severity — never content, paths, filenames, username, or ATR version. Regex layer structurally misses paraphrased/semantic attacks; the page's own recommendation is to treat ATR as the content layer and pair it with credential brokering, sandboxing, and human-in-the-loop.