Configuring your AI vulnerability harness, Part 2: The steering file

What it claims

A persistent instruction file, not a better model, is what turns a frontier LLM's vulnerability triage from narrative guesswork into reproducible, evidence-cited findings. The authors report that about 30% of unsteered findings referenced code structures that did not exist in the target repo: entirely fabricated paths, not misreadings. With the "verify before reporting" instruction, they saw zero fabricated paths in their testing. The full file is published in a companion GitHub repository.

How it works

The details the summary omits but that matter if you adapt the file:

• Control-to-class mapping. Each control applies only to the vulnerability class it addresses. For example, AWS WAF SQLi rules in blocking mode are a STRONG control (0.25x) against SQL injection and irrelevant to deserialization. Attack-blocking multipliers stack multiplicatively down to the 0.15 floor.
• Access controls versus attack-blocking controls. The two are scored differently:
• Authentication and authorization reduce only the reachability component.
• WAF rules and input validation reduce the full score.

The first version applied auth as a blanket multiplier, which pushed everything behind IAM to P2. After the fix, a command injection behind IAM is P0.

• Three different numbers. The priority tiers are compared against the final score: the base score after attack-blocking multipliers and any threat-intel boost. That is neither the confidence value nor the scanner's severity, so don't conflate them when tuning.
• Threat-intel signals. These are CISA KEV (including ransomware linkage), EPSS (30-day exploit probability) and the existence of a public PoC. The authors treat them as raising urgency, not severity.

Evidence

• Taint as the top signal. On known-vulnerable apps, findings with confirmed taint and no sanitization were exploitable 89% of the time. Findings the model self-rated HIGH without structural confirmation were exploitable 34% of the time. This is the stated justification for the 0.30 taint weight.
• 10-vulnerability test app. The app had eight exploitable vulnerabilities and two mitigated by WAF, VPC-isolated Lambda and IAM.
• Unsteered: the model found all 10 and also ranked the two mitigated ones lower. The difference was that its severity came from intuition ("command injection, so CRITICAL") and it did no structural verification.
• Steered: the model found 9 of 10. It skipped a hardcoded credential that was defined but never called in unit-test code. It moved both mitigated findings to P3 with written rationale, showed the confidence math for each finding, and verified each claimed dataflow.

So the gain on this app was auditability and calibrated priority, not recall. The 30%-fabrication figure comes from earlier development, not from this test.

How to use it

Installation options:

• Always-on (Kiro): put the file at .kiro/steering/vuln-triage.md. Use ~/.kiro/steering/ to apply it to every workspace.
• On demand (Kiro skill): create a folder containing a SKILL.md whose front-matter name matches the folder name. Put it in .kiro/skills/ or ~/.kiro/skills/. It is invoked as /vuln-triage or auto-matched by its description. This keeps the instructions out of the context window for unrelated turns.
• Custom agents: steering files and skills are not loaded automatically by custom Kiro agents. Add them to the agent's resources yourself.
• Claude Code: the repo ships a CLAUDE.md version.

Recalibrate the weights against your own labeled true-positive and false-positive set. The authors call the formula a starting point.

Limits

The authors state the file provides methodology, not machinery:

• Dataflow is approximated. The model reads code to infer dataflow. Deterministic AST taint tracking (e.g. Tree-Sitter-based) is suggested for Python. For other languages the model's result is "reasonable but not authoritative".
• No live infrastructure checks. It parses IaC only, so it cannot confirm that a WAF is actually attached to API Gateway or that a security group hasn't drifted.
• No live threat-intel feeds. KEV knowledge comes from training data. Current exploitation status needs an API integration.
• No exploit confirmation. Confirming exploitability requires a sandbox, request signing and differential testing.
• Small validation. It covers a single purpose-built app with 10 vulnerabilities. The "do not file P3" rule is a deliberate trade of recall for engineer trust.