arxiv.orgpaper

AI Agents May Always Fall for Prompt Injections (Abdelnabi & Bagdasarian)

This arXiv paper by Abdelnabi and Bagdasarian argues that the dominant prompt-injection defense paradigm — data/instruction separation — fails to catch attacks built on contextual manipulation and also breaks legitimate agent behavior. The authors reframe prompt injection through Contextual Integrity (CI), a privacy theory judging whether information flows comply with contextual norms, and construct scenarios where attackers misrepresent flows, manipulate norms, or mix multiple flows to bypass defenses. They derive an impossibility result: any adversary can construct a context making a blocked flow look legitimate, or any defender who tightens norms enough will block genuinely legitimate flows, and propose CI-aware alignment as a principled path forward for frontier autonomous agents.