ADLC metrics: How to measure agentic development security | Endor Labs

What it is

An Endor Labs guide (a vendor page that also promotes its AURI product) defining formulas for measuring security when AI coding agents write much of the code. It splits measurement into two surfaces: agent output (insecure code, vulnerable dependencies, leaked secrets) and agent behavior (shell commands, file reads, tool calls, package installs). "ADLC" here means the agentic development lifecycle, not IBM's agent development lifecycle for building agents as products.

How it works

The page argues that PRs scanned, total findings and scan pass rate scale with agent volume, not risk. It recommends dividing confirmed findings by merged PRs, or by changed lines divided by 1,000. Per merged change suits policy and process metrics. Per changed lines suits agents producing changes of very different sizes. Pick one per metric and keep it stable.

Formulas the page gives:

• Discovered vs approved: approved components in use ÷ discovered components in use, computed separately for agents, models, MCP servers and skills. A 0.6 ratio for MCP servers means 40% are unreviewed.
• Permission review coverage: MCP servers and skills with a recorded permission review ÷ total in use.
• Pre-execution split: risky actions stopped before execution ÷ (stopped before + detected after). Risky categories: destructive shell commands, sensitive file reads (.env, credentials, SSH keys), credential access or exfiltration, installs from unapproved sources.
• Policy block vs model refusal: record them separately. Only deterministic blocks are auditable.
• Agent-generated code: confirmed findings introduced by agent-authored changes ÷ merged agent-authored changes, using diff-based analysis so legacy debt is excluded, and compared with a human baseline on similar repos.
• Evidence-adjusted share: new findings with reachability or exploitability proof ÷ total new findings. Track false positive rate (closed as false positive ÷ triaged) beside it.
• Secrets: confirmed live secrets per period, split by agent vs human, plus the share caught at pre-commit.
• Packages: risky or nonexistent installs blocked before execution ÷ total such attempts, because install scripts run before post-install scans. Also reachable ÷ total vulnerable dependencies, and unapproved dependency introduction rate.

Supporting action metrics: policy coverage across agents, blocks per 100 agent sessions by category, prompt-injection test success rate on a fixed suite, and regression rate after model, prompt or policy changes.

What to do

• Tag agent-authored changes with commit trailers, PR labels or agent telemetry. For mixed changes, use a rule such as "agent-authored if the agent produced the majority of changed lines."
• Pair each security metric with a flow metric: blocked PR rate, fix acceptance rate, and median time to fix reported per severity tier.
• Choose six to eight metrics across the five categories, baseline them for a month, then set targets by repository risk tier.
• Review weekly: pre-execution stops, packages blocked, blocked PR rate, confirmed findings per change. Review monthly: policy coverage, evidence-adjusted share, time to fix. Report quarterly trends to leadership.
• Collect data where agents act (hooks, tool-call logs), since post-merge data cannot show pre-review behavior.
• Track suppression rate, because the evidence-adjusted share can be inflated by suppressing findings.

Caveats

The scorecard numbers are hypothetical and the page says they are not benchmarks. The figures of 83% fewer blocked PRs and up to 95% false positive reduction for AURI are Endor Labs' own reported results, not industry benchmarks. The page cites the DORA 2024 report (AI adoption associated with lower delivery stability) and the USENIX Security 2025 paper by Spracklen et al. on package hallucination, but gives no figures from either. It also notes that pattern-based rules often miss business logic flaws such as IDOR and broken authorization.