Skip to content

Back to the catalog

Configuring your AI vulnerability harness, Part 2: The steering file
aws.amazon.comarticle

Configuring your AI vulnerability harness, Part 2: The steering file

AWS Security Blog's second post in a series on building an "AI vulnerability harness" explains the configuration (steering file) that stops an AI model from hallucinating vulnerabilities and inflating severity during security code review. It details five configuration sections — structural verification of findings, an evidence-based confidence formula (weighting confirmed taint flow at 0.30), infrastructure-aware severity multipliers for controls like AWS WAF and IAM, CISA KEV/EPSS threat-intel boosting, and a four-tier priority classification — and reports validation numbers (e.g., unsteered models fabricated code paths in ~30% of findings; steered confidence of HIGH without structural confirmation was only 34% exploitable vs 89% with confirmed taint). The steering file is shipped as a Kiro steering file, a Kiro skill, and a CLAUDE.md for Claude Code, with full source in a linked GitHub repo.

TL;DR AWS details a concrete "steering file" methodology (evidence scoring, infra-aware multipliers, KEV/EPSS boosts) that measurably cuts hallucinated findings and misprioritized severity in AI-driven vulnerability triage.
Also seen from another angle🧠 AI →
Document

What it is

The steering file itself—a methodology document for configuring AI coding assistants (Kiro, Claude Code, Cursor, GitHub Copilot) to perform vulnerability triage without hallucinating findings. Shipped as a Kiro steering file, a Kiro skill, and a CLAUDE.md, with source in AWS's companion GitHub repo.

How it works

The file encodes control-to-vulnerability mappings explicitly rather than letting the model guess. Example given: an AWS WAF with SQL injection rules in blocking mode gets a 0.25x multiplier against SQLi specifically, but is irrelevant to deserialization attacks. Attack-blocking controls (WAF, input validation) stack multiplicatively with a floor of 0.15—never fully zeroing out risk. Access controls (IAM, auth) are scored separately from attack-blocking controls: authentication reduces the reachability score component, not the confidence score, because it limits who can attempt exploitation, not what happens once they do. This separation required one correction iteration—the initial version wrongly applied authentication as a blanket multiplier, suppressing all IAM-gated findings to P2 regardless of actual impact.

Preconditions / Affected

Applies to teams running AI-assisted security code review where the model has access to IaC files (for control mapping) and the target codebase (for structural verification). Requires a tool that loads persistent repo instructions (Kiro, Claude Code, Cursor, Copilot).

What to do

  • Drop the file at .kiro/steering/vuln-triage.md for always-on loading by Kiro's default agent, or ~/.kiro/steering/ for machine-wide use.
  • For on-demand use instead of persistent context consumption, place as a skill folder with SKILL.md under .kiro/skills/; it activates as /vuln-triage or automatically via description matching.
  • Custom Kiro agents don't load steering/skills automatically—must be added explicitly to the agent's resource list.
  • For non-Kiro tools, use the shipped CLAUDE.md or adapt the same content to your tool's instruction-file convention.
  • Calibrate the confidence-formula weights against your own labeled dataset of true/false positives rather than using the 0.30 taint weight as-is.
  • Enforce the "do not file P3 findings as tickets" rule to avoid training engineers to ignore the pipeline's output.

Caveats

Validation was on one purpose-built test app (ten vulnerabilities, eight exploitable, two infra-mitigated)—steered model missed one (an unreachable hardcoded test credential, never called). The file explicitly does not replace: programmatic/AST-based taint tracking (model only approximates dataflow by reading code), live infrastructure verification (can't confirm deployed WAF/security-group state via API), live threat-intel feeds (model's KEV/EPSS knowledge is training-data-bound, not real-time), or execution-based exploit confirmation.

Techniques named here