Engineering a security harness for AI coding agents | Blog | Endor Labs

What it adds beyond the summary

The summary names the seven-question framework and the 91.7% token figure. What it omits, and what an engineer evaluating this design would act on, is the internal structure of three of those questions plus the benchmark's caveats.

The three-layer control model

The page's central operational distinction (question 4) is a three-layer scheme the summary flattens into "hard enforcement gates":

• Instruction guides — the prompt tells the model what evidence to use and when to request approval.
• Validation rejects — checks whether required evidence, provenance, risk decisions, or approval state are missing.
• Enforcement withholds — the runtime does not expose or execute consequential capabilities until conditions are met.

The stated principle: not every run need be deterministic (that would defeat using a model), but "every acceptable run must satisfy the same rules" — schemas, evidence prerequisites, approval gates, tool permissions, budgets, stop conditions. Consequential workflows should behave like state machines with budgeted loops: "A loop can recover. It cannot skip a gate."

The evidence pattern (question 2) is likewise concrete: Endor Labs Agent Kit uses resolve scope → retrieve evidence → reason → record gaps, with deterministic systems (not the model) establishing versions, findings, reachability, upgrade impact. Missing evidence is a returned result, not license to guess.

Evidence and its caveats

The benchmark: across 34 prompts, 13 scopes, 884 responses, same model and step limits, the evidence-equipped agent used 6.6M vs 79.5M tokens (91.7% fewer) than agents reconstructing posture via reconnaissance loops. The page explicitly qualifies this: it "measures execution economics, not correctness," excludes the deterministic upstream computation from per-question cost, and gains vary by task/model/environment. An engineer citing this number should carry those caveats.

Separately, a deliberately simple demo — upgrading Jinja2 3.1.1 → 3.1.6 in one manifest — where the success condition was not "found the upgrade" but "stopped at the right boundaries" (approval before edit, then again before opening the PR).

Operational specifics the summary compresses

• Evaluation at three levels (question 5): Process (did it follow the evidence path/gates?), Result (was the fix correct/complete?), Outcome (did risk actually drop without regressions?) — plus contestability: a reviewer must be able to see the evidence, challenge, reject, or rerun. "A human clicking 'approve' without enough information to disagree is not meaningful human oversight."
• Failure containment (question 6): fail closed when proof is absent; treat repo contents and tool output as data, not authority ("untrusted text never becomes authority"); make retries idempotent and checkpoint state so interruptions don't leave ambiguous states.

What this is

A vendor design essay (Endor Labs, promoting Agent Kit), not a paper or release — no code, weights, or reproducible artifact. Its value is the control framework and the token benchmark; treat it as an architecture argument to evaluate against, not something to install.