Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files

What it is

PackHallu attacks the coding-rule-file ecosystem (.cursorrules, AGENTS.md, CLAUDE.md) by injecting an optimized malicious prompt into an otherwise benign, shared rule file. When a victim downloads the poisoned file and runs a coding agent (Claude Code, Cursor, Codex, etc.), the agent is steered to import an attacker-chosen package (pm) in place of the legitimate one (pv) — e.g., numpy_hl instead of numpy. The attacker separately publishes pm to PyPI, so the swap becomes a working supply-chain compromise once the generated code is installed and run.

How it works

PackHallu is an evolutionary search over candidate injected prompts, run entirely against a locally deployed surrogate agent — the attacker never needs access to the real victim model, framework, or system prompt. Two mechanisms make the search tractable:

• Trajectory-Level Signal: instead of scoring only the agent's final generated code (sparse: pass/fail per full multi-turn run), PackHallu scores the surrogate agent's entire reasoning trajectory, giving much denser optimization feedback per iteration.
• Self-Attributed Refinement: an "attack LLM" critiques why a candidate prompt failed to induce the swap and proposes a revision strategy, which generates "offspring" prompts for the next round — directly countering the dilution effect of the malicious prompt sitting inside long, legitimate rule-file content.

flowchart LR
    A[Surrogate agent + task set] --> B[Candidate prompt]
    B --> C[Run on surrogate, capture trajectory]
    C --> D[Trajectory-level score]
    D --> E{Induces package swap?}
    E -->|No| F[Self-Attributed Refinement: critique + revise]
    F --> B
    E -->|Yes, best so far| G[Optimized malicious prompt p*]
    G --> H[Published poisoned rule file]
    H --> I[Victim agent: different framework/LLM]
    I --> J[Transfers: still swaps pv -> pm]

Preconditions / Affected

• Applies to any agentic coding setup that consumes shared/downloaded rule files (tested across 8 agent frameworks, 13 backbone LLMs, 3 benchmarks covering 10 widely-used Python packages).
• No knowledge of the victim's system prompt, backbone LLM, agent framework, or actual user task is required by the attacker — only a benign rule file to poison and a surrogate agent to optimize against.
• Attack is specific to a given victim-package/malicious-package pair, not a generic payload.

Caveats

• The paper states PackHallu "substantially outperforms" heuristic- and prior optimization-based injection methods in their own evaluation, but gives no concrete attack-success-rate numbers in the fetched text — only "high" and "strong transferability" qualitatively.
• Detection: the authors tested state-of-the-art prompt-injection detectors against PackHallu-crafted rule files and found they "either miss most attacks or suffer from high false positive rates" — no detector configuration or threshold details are given, so this should be read as a gap finding, not a validated defense to deploy.