Skip to content

Back to the catalog

AI Security Briefing
arxiv.orgpaper

AI Security Briefing

This paper presents the first systematic evaluation of package-install-time supply-chain attacks delivered through ordinary project-setup documentation (README, requirements files, Makefiles) against production AI coding-agent harnesses. Testing frontier models across twelve scenarios in five attack classes grounded in real incidents, the authors find install-time security depends on the harness-model pairing rather than the model alone: agents reliably catch blatant typosquats but miss plausible separator-confusion names (e.g. azurecore for azure-core) and almost universally miss registry-redirection attacks, with the source blind spot recurring on npm and Cargo. A deterministic pre-install check that verifies package names, sources, and versions before code execution closes most of the gap.

TL;DR Editing a README or requirements file can silently steer AI coding agents into installing malicious or vulnerable dependencies, and detection varies wildly by harness-model pairing.
Document

What it is

"Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents" (arXiv, submitted 16 Jul 2026) — the first systematic evaluation of package-install-time supply-chain attacks delivered through ordinary project-setup documentation. The summary covers the core finding; below are the two operational points it does not carry.

What to do

The paper reports a hierarchy of mitigations, and the ordering matters for anyone relying on them:

  • Security-oriented prompts help only the dimension they name. Telling the agent to watch for typosquats does not improve source-based detection; prompting recovers part of the gap but only along the axis it explicitly calls out. Do not treat a "be careful about dependencies" instruction as broad coverage — it protects the one thing it names and nothing else.
  • **A deterministic pre-install check that verifies names, sources, and versions before any code runs closes most of the gap.** The timing is the point: the check must gate execution, not run after the agent has already invoked the installer.

Preconditions / Affected

  • The scenarios are grounded in documented incidents, not hypotheticals — the attack classes correspond to real supply-chain events.
  • The source blind spot recurs on npm and Cargo, where nearly every model installs the untrusted dependency. Name-based detection, by contrast, carries over less consistently across ecosystems — a harness that catches a bad name in one package manager may not in another. Do not assume name-detection results transfer between ecosystems; source-detection failure, however, appears to generalize.

Caveats

The prompt-based mitigation is explicitly partial and axis-specific (see above). The strong result is the deterministic pre-install gate, not model behavior or prompting.

flowchart TD
    A[Setup doc: README / requirements / Makefile] --> B{Pre-install check<br/>names, sources, versions}
    B -->|verified| C[Install proceeds]
    B -->|fails| X[Blocked before code runs]
    A -.no check.-> D{Model + harness only}
    D -->|blatant typosquat| C
    D -->|separator-confusion name| E[Often installed]
    D -->|registry redirection| F[Almost always installed<br/>npm / Cargo]

Technologies named here

Techniques named here

  • Registry redirection
  • Typosquatting
  • Separator-confusion name attack
  • Known-vulnerable version installation