Understanding and Mitigating Role Spoofing in LLMs is an arXiv paper (October 2026) on role spoofing, a prompt injection where malicious tool output impersonates the user. It shows models track "who said what" using both content cues (semantics, style) and structure cues (control tokens, formatting). Attacks combining fake structure and content succeed often, adversarial training stops only content-based attacks, and linear probes trained to predict conversation roles detect spoofing across settings.
LLM
role spoofingrole confusionprompt injection+1
AI Security· Added 31 Oct 2026, published 29 Oct 2026
SEAL is a defense against indirect prompt injection for local LLM agents. It learns a soft prompt and scoring readout offline, then uses a frozen task model to score and delete injected tokens from tool outputs, with queries built from trusted initial context. On AgentDojo, attack success falls from 34.6% to 0.1% and Joint rises from 51.4% to 77.5%, using about 127x fewer defense tokens than PISanitizer.
LLMAgentDojoAgentDyn+1
indirect prompt injection
AI Security· Added 31 Oct 2026, published 29 Oct 2026
Paper studying whether self-evolution weakens personal agents' robustness to indirect prompt injection (IPI). A two-phase method has the user ask the agent to follow tasks in external content, then tests with a third-party injected task the user did not authorize. Against the non-evolving base agent, attack success rose 14.0 to 48.8 points under skill evolution and 3.0 to 39.3 points under reinforcement learning. Code is released.
indirect prompt injection
AI Security· Added 31 Oct 2026, published 29 Oct 2026
Log-To-Leak is an arXiv paper (cs.CR, cs.AI) by Yuepeng Hu, Chongyu Fan, Sirat Samyoun and Jian Du on prompt injection attacks against tool-using LLM agents through the Model Context Protocol. The title points to a logging-based data leakage channel. The fetched page holds only the title, authors and arXiv link, with no abstract, so the method and results are not visible here.
Model Context ProtocolLLM
Log-To-Leakprompt injection
AI Security· Added 29 Oct 2026, published 28 Oct 2026
Heads of Security (HoS) is a prompt injection defense that enforces instruction-data separation with learned per-head attention gates, redistributing attention between trusted and untrusted tokens. Evaluated on Llama-3.3-70B-Instruct and GPT-OSS-20B against SecAlign++ and ASIDE. The black-box RL-Hammer attack reaches 97% ASR on GPT-OSS-20B and 85% on SecAlign but 10% on HoS. The white-box Claudini attack needs 3 times more FLOPs on HoS. HoS keeps base-model utility in few-shot settings, where SecAlign and ASIDE degrade.
LlamaGPT-OSS-20BLLM
Heads of Securityprompt injectionSecAlign+++4
AI Security· Added 28 Oct 2026, published 27 Oct 2026
Heads of Security (HoS) is a prompt injection defense that trains a small set of per-head attention gates to shift attention between trusted and untrusted tokens, separating instructions from data. Tested on Llama-3.3-70B-Instruct and GPT-OSS-20B, the black-box RL-Hammer attack reaches 10% ASR on HoS versus 97% on base GPT-OSS-20B and 85% on SecAlign. HoS keeps base utility in few-shot settings, where SecAlign and ASIDE degrade.
TransformerLlamaLLM
Heads of SecuritySecAlign++SecAlign+5
research, training· Added 28 Oct 2026, published 27 Oct 2026
What Victims Take Away is an October 2026 arXiv paper analysing 563,349 Tensor Trust attack records to see how players reuse prompt injections they received. Within a week, targets reused 10.0% of attacks that broke their defense, versus 0.03% of matched attacks. Breaches were followed by defense changes that cut same-family attack success by 24.5 points. Received texts made up 23.2% of successful hijacks. The authors propose that AI security tools describe techniques without disclosing full attack text.
Tensor TrustLLM
prompt injectionpayoff-biased social learningvictim-offender overlap
AI Security· Added 28 Oct 2026, published 27 Oct 2026
Log-To-Leak is a prompt-injection attack on tool-using LLM agents that abuses Model Context Protocol tool metadata to trigger an attacker-controlled logging call after the last legitimate call, leaking the user query, server name, server response and agent response. Evaluated on five MCP servers and four agents, it reaches 62.64-100% attack success with 76.55-95.64% logging accuracy. Three defenses leave over 55% residual ASR on GitHub MCP with GPT-4o.
Model Context ProtocolGitHubGPT-4o+1
Log-To-Leaktool poisoningprompt injection
AI Security· Added 28 Oct 2026, published 26 Oct 2026
An empirical study of security concerns in the OpenClaw AI personal assistant framework, using 25K+ posts from 13 subreddits (162 security-relevant) and 358 security-tagged GitHub issues (180 retained). It maps 521 CWE instances across 59 weakness types, concentrated in access control, sensitive-data exposure, injection and unsafe execution. It extracts 543 practitioner mitigations into seven strategies and reconstructs a sensitive-data exposure path with dummy secrets.
OpenClawGitHubLLM
prompt injectionCWEtrust boundaries+1
AI Security· Added 28 Oct 2026, published 26 Oct 2026
Scope-Gap Injection (SGI) is an indirect prompt injection method for multi-stage LLM-agent safety pipelines, tested in the IJCAI 2026 OpenClaw Agent Security Attack Competition. The visible task and skill stay benign while target identifiers, bypass parameters and authorization steps sit in an attachment written as a business worklist, which the S2 LLM judge appears not to inspect. SGI bypassed all three stages (static scan, LLM judge, live audit) for 5 of 19 targets, best F = 0.385. Attachment placement had 78% S2 acceptance versus 23% in-skill. The paper closes with defenses that align evaluator scope with agent channels.
NTBench (Near-Target Benchmark) tests LLM and agent robustness to near-target prompt injection, where injected instructions steer a response toward a plausible but incorrect or harmful outcome instead of abandoning the task. It builds reference-guided target-instruction pairs on existing datasets, with human validation, and evaluates seven LLMs on ten language datasets and two agent benchmarks. Attack success rose 9.3 to 35.8 points (incorrect targets) and 1.7 to 17.3 points (harmful targets) over fixed-target baselines.
Paper on prompt injection delivered through forensic evidence, where the seized device's contents are attacker-controlled and the examiner must read them. It extends the anti-forensic model to cover manipulation of the reasoning layer and gives a taxonomy (objective, technique, payload location, concealment). Three proof-of-concept attacks on an LLM-assisted analysis pipeline change a reported conclusion, write attacker-chosen content to the analyst workstation, and trigger outbound network traffic. Mitigations are proposed.
MPIDetector is a detection framework for multi-modal prompt injection that uses few-shot in-context learning so an LLM can reason about the semantic intent of text and image inputs, without fine-tuning. The authors built a benchmark of eight textual and seven visual attack categories plus benign sets. They report up to 90.91% accuracy on text, 99.28% on images, and up to 99.80% detection on 500 obfuscated WebInject web-agent injections, beating baselines.
GeminiQwenLLM
prompt injectionWebInjectMPIDetector
AI Security· Added 24 Oct 2026, published 21 Oct 2026
Paper (arXiv, 21 October 2026) on structural false negatives in LLM vulnerability discovery harnesses. Starting from Knostic's OpenAnt and raptor, then 10 more open-source harnesses, the authors map 13 points where code selection, filtering, and finding storage or merging drop vulnerabilities. One file was dropped by six harnesses before the model saw it, five without notice. It covers attacker evasion, content-steered verdicts, prompt injection, and design principles and tests.
OpenAntraptorLLM
KnosticPtacek and Newshamprompt injection+1
AI Security, AppSec· Added 24 Oct 2026, published 21 Oct 2026
Paper on memory injection, where prompt injection persists across sessions through the long-term memory of LLM-as-a-service platforms. It presents a general threat model and shows all NIST attack goals are achievable on ChatGPT Instant 5.3, Claude Sonnet 4.6 and Gemini 3 Flash. Vectors include personal memories, projects, skills, organization instructions and MCP resources. On Claude, shared memory lets an attacker compromise multiple users in an organization and exfiltrate prompts. Nearly half of the derived mitigations are unimplemented.
ChatGPTClaudeGemini+2
memory injectionprompt injectionNIST+2
AI Security· Added 24 Oct 2026, published 20 Oct 2026
CarryoverBench is a benchmark of 12 personal-assistant scenarios and 101 attack tasks for carryover attacks, where a one-time indirect prompt injection planted in agent memory files or scheduled tasks re-fires in later sessions. Three trigger channels (Instant, Autonomous, Latent) are scored by a rule-based verifier with separate planting and triggering stages. Across five frontier models, payloads re-fire over five rounds without measurable decay. Porting the Latent channel to the Hermes runtime weakens the effect but does not remove it.
nokast-secureRAG is a roughly 0.5B-parameter small language model that works as a semantic firewall between retrieval and generation in local RAG systems. It classifies each query and context pair as safe, suspicious or malicious. It reaches 0.994 recall at a 0.026 false-positive rate in-distribution, beating Llama Guard 3 1B and Prompt-Guard-86M (0.900 recall, 0.690 FPR). Recall on PoisonedRAG falls to near zero, and single-request latency is 293-404 ms.
Fake News, Real Crashes is an October 2026 arXiv paper presenting a benchmark for information injection attacks on LLM financial trading agents. It spans five injection vectors, seven event categories and about 10,000 decisions on 15 U.S. stocks. Fabricated news reached 62% attack success versus 14% for prompt injection, and two 30-word templates hit 100%. Only hierarchical aggregation reduced success among the defenses tested.
LLM
information injectionprompt injectionfact poisoning+1
AI Security· Added 21 Oct 2026, published 19 Oct 2026
SpanGuard is an indirect prompt injection (IPI) guardrail that splits detection into two steps. Lightweight unsupervised segmentation first locates candidate instruction blocks in tool outputs, then a small LLM judges whether each block is authorized by the user's request. The authors report a 0.32% attack success rate with negligible benign utility loss, and the guardrail still works with substantially smaller models.
LLM
indirect prompt injectionprompt injection
AI Security· Added 21 Oct 2026, published 19 Oct 2026
PIAtlas is a taxonomy of prompt injection attacks organized by how the injected instruction subverts the model, with six mechanism classes as roots of a technique and sub-technique hierarchy. The authors release PICorpus, a license-clean collection of public prompt-injection datasets, and map thousands of records to PIAtlas under a human-verified protocol. Results show attacks combine several techniques from few mechanisms, agent benchmarks cover only part of the space, and adaptive attackers favor mechanisms that curated benchmarks underrepresent.
LLM
PIAtlasPICorpusprompt injection
AI Security· Added 21 Oct 2026, published 18 Oct 2026
Ignored, Not Refused audits 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo. A label-blind LLM judge finds most failed indirect prompt injections are ignored (76-80%) rather than refused (17-18%). Changing only the system prompt moves InjecAgent ASR from 0.2% to 43%. The Attack-Task Outcome Matrix (ATOM) shows 21.7% of AgentDojo episodes were safe but left the user task incomplete. The corpus, judge prompt and code are released.
LLM
indirect prompt injectionAgentDojoInjecAgent+2
AI Security· Added 20 Oct 2026, published 16 Oct 2026
R^2ECAP (Reused Reasoning-Cache Poisoning) is a black-box attack that plants poisoned reasoning traces through a chat interface, exploiting LLM systems that reuse KV caches of reasoning. A preceding trace steers the model, and the retained trace keeps residual adversarial influence in its cache after the preceding trace is discarded. On three open-weight LLMs and three datasets it reaches up to 79% ASR, beats the white-box HijackKV in 8 of 9 settings, and reaches 46% ASR in a reasoning-trace retrieval system. Only 1% of retained responses are flagged by a popular prompt injection detector.
LLM
R^2ECAPHijackKVprompt injection
AI Security· Added 20 Oct 2026, published 16 Oct 2026
Evaluates a native probabilistic decision model (Jev), Llama Prompt Guard 2 (22M and 86M) and an LLM judge on 47,611 held-out examples covering direct/indirect prompt injection, hard-benign inputs, obfuscation and distribution shift. The LLM judge has the best AUROC (0.897), followed by Jev (0.871), Prompt Guard 2 86M (0.757) and 22M (0.714). Jev's native probabilities do not consistently beat post-hoc calibrated classifier scores, and temperature scaling improves calibration but not discrimination. At a 1% false-negative-rate target, every system needs over 91.8% escalation, so probabilistic outputs alone do not give reliable security decisions under distribution shift.
LlamaLLM
JevPrompt Guard 2prompt injection+1
AI Security· Added 17 Oct 2026, published 14 Oct 2026
INFUSE is an end-to-end benchmark for indirect prompt injection (IPI) across long-horizon agent workflows (shell commands, code changes, dependency installs, remote services). It has 40 realistic tasks and a taxonomy of 56 attack scenarios in five threat categories, with attacks injected via interaction surfaces encountered mid-task. Across 19 LLMs (GPT, Claude, Gemini, Llama, Gemma, Qwen) and 41,824 adversarial trajectories, mean attack success was 19.6% (LLM-judged) and 15.4% (heuristic), per-model 3.2–41.9%; guard models gave limited protection, while prompt-level hardening cut mean ASR from 58.2% to 14.3%.
LLMClaudeGemini+3
indirect prompt injectionINFUSE
AI Security· Added 17 Oct 2026, published 14 Oct 2026
Introduces SA-CUA, a dual-LLM framework that gives Computer Use Agents a trust boundary against indirect prompt injection. It derives conservative deterministic rules from the accessibility tree to unmask trusted screen regions for the privileged LLM, and adds an endorsement channel that moves values from untrusted to trusted domains by verification. Human input is needed only at task specification. It reports much higher utility than CaMeL-CUA with a zero attack success rate.
LLM
SA-CUACaMeL-CUACaMeL+2
AI Security· Added 15 Oct 2026, published 13 Oct 2026
Introduces SA-CUA, a dual-LLM framework for Computer Use Agents (CUAs) that defends against indirect prompt injection. It builds a trust boundary by deriving conservative deterministic rules from the accessibility tree, which unmask trusted screen regions in the privileged LLM's screenshot and keep untrusted content separate. An endorsement channel lets values move from the untrusted to the trusted domain through verification over trusted channels. This removes the need for per-action user confirmation and the confirmation fatigue it causes. SA-CUA is reported to keep much higher utility than CaMeL-CUA with a 0% attack success rate.
LLM
SA-CUACaMeL-CUACaMeL
agents, research· Added 15 Oct 2026, published 13 Oct 2026
BrowseSafe++ extends BrowseSafe to 38,675 paired attacked/clean web pages, most with prompt injections observed in the wild on production sites. Pages cover twelve site types, 24 attack types and 18 injection locations, each given as HTML, Markdown, accessibility tree and screenshot with a user task and expected answer. The authors train BrowseSafe-Guard, two 0.8B detectors (text and screenshot) that beat a frontier-model-based detector on F1 with low false positives and low inference cost. BrowseSafe-Adapt rewrites injections using browser-agent feedback, raising attack success from a few percent to up to 95.41%.
LLM
BrowseSafe++BrowseSafeBrowseSafe-Guard+3
AI Security· Added 15 Oct 2026, published 12 Oct 2026
StableJudge is a defense framework against score-hijacking prompt injection in LLM-as-a-Judge pipelines (e.g. paper review, recommendation). A compact 0.13B "Sentry" model localizes injected spans and sanitizes them before the judge scores the content, trained on adversarially evolving attacks, and deterministic fine-grained scores are derived from the judge's output distribution. It improves score recovery under attack on held-out scientific abstracts and out-of-distribution attack variants while preserving general performance, decoupling safety from judge model scale. Code and data are promised but not yet released.
LLM
prompt injectionindirect prompt injection
AI Security· Added 15 Oct 2026, published 12 Oct 2026
Paper testing whether an LLM agent operating life support in the NASA-derived BioSim lunar habitat simulation can be induced by delayed prompt injection, hidden in truthful S-band telemetry, to shut down life support and hold the shutdown while the crew suffocates. It compares Claude, GPT and Grok configurations: Claude mostly refuses and cites the living crew, GPT shows some susceptibility, and Grok given a "trusted commission" framing more often cuts life support. It also varies the framing of the message (trusted commission, peer, untrusted source) and uses a harmless-procedure-drop attack as a proxy for whether life is valued intrinsically. Code and traces are open-sourced.
ClaudeGrokLLM
BioSimNASAderived-action trap+1
AI Security· Added 15 Oct 2026, published 10 Oct 2026
Introduces Learnable Trust-Boundary Delimiters (LTBD), a defense against prompt injection that uses a few learnable delimiters to mark trusted user instructions versus untrusted external data, without changing LLM parameters. It reports 0.00% ASR on AlpacaFarm and 0.11-0.19% on TaskTracker, outperforming inference-time defenses and rivaling training-based ones with negligible overhead. It also holds up against adaptive attacks where the adversary knows the defense.
LLM
LTBDAlpacaFarmTaskTracker
AI Security· Added 9 Oct 2026, published 8 Oct 2026
arxiv.orgpaper🛡️ Security
