Skip to content

215 results for “Prompt Injection”

ListGrid
What you seeeverything
RecentOldestAddedA–Z
All types215Papers59Repos49Articles42Write-ups30Threads12Advisories11
Talks3Tools3Videos3Courses2Other1
“Prompt Injection”
x.comthread🛡️ Security

Tanmay (@TanmayK_03) on X

DBHub, a database MCP server, had a read-only mode bypass in its SQL check. The check treated every `--` as a comment, but MySQL only starts a comment when whitespace follows it. A hidden write statement could therefore still run, and a prompt injection could trigger it. Fixed in version 0.22.6. Replies say enforcement belongs in database grants and that parser tests need to be dialect-specific.

DBHubMySQLModel Context Protocol
prompt injection

AppSec, AI Security· Added 31 Oct 2026, published 30 Oct 2026

arxiv.orgpaper🛡️ Security

Understanding and Mitigating Role Spoofing in LLMs

Understanding and Mitigating Role Spoofing in LLMs is an arXiv paper (October 2026) on role spoofing, a prompt injection where malicious tool output impersonates the user. It shows models track "who said what" using both content cues (semantics, style) and structure cues (control tokens, formatting). Attacks combining fake structure and content succeed often, adversarial training stops only content-based attacks, and linear probes trained to predict conversation roles detect spoofing across settings.

LLM
role spoofingrole confusionprompt injection+1

AI Security· Added 31 Oct 2026, published 29 Oct 2026

securityweek.comarticle🛡️ Security

Google Touts Powerful Cybersecurity Capabilities in Gemini 4 Carbon Model

Gemini 4 Carbon is a Google frontier model announced with restricted access for cyber defenders through the Fairwind Program (650+ organizations). Google reports 88.2% on SEC-bench Pro, 54.8% on ExploitGym and 71.0% on CWE-bench, ahead of GPT-6 Astra and Claude models, with no guardrails. It also cites prompt injection and cheating-resistance results, and pricing of $2/$10 per million tokens.

GeminiGPT-6 AstraWiz+1
Gemini 4 CarbonGemini 4 ArgonFairwind Program+6

AI Security· Published and added 29 Oct 2026

arxiv.orgpaper🛡️ Security

SEAL: Sealed Queries for Local Sanitization against Indirect Prompt Injection

SEAL is a defense against indirect prompt injection for local LLM agents. It learns a soft prompt and scoring readout offline, then uses a frozen task model to score and delete injected tokens from tool outputs, with queries built from trusted initial context. On AgentDojo, attack success falls from 34.6% to 0.1% and Joint rises from 51.4% to 77.5%, using about 127x fewer defense tokens than PISanitizer.

LLMAgentDojoAgentDyn+1
indirect prompt injection

AI Security· Added 31 Oct 2026, published 29 Oct 2026

arxiv.orgpaper🛡️ Security

Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt Injection

Paper studying whether self-evolution weakens personal agents' robustness to indirect prompt injection (IPI). A two-phase method has the user ask the agent to follow tasks in external content, then tests with a third-party injected task the user did not authorize. Against the non-evolving base agent, attack success rose 14.0 to 48.8 points under skill evolution and 3.0 to 39.3 points under reinforcement learning. Code is released.

indirect prompt injection

AI Security· Added 31 Oct 2026, published 29 Oct 2026

x.comthread🛡️ Security

Rohan Paul (@rohanpaul_ai) on X

Rohan Paul's X post summarizes the paper "Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol", from TikTok researchers. It reports that MCP-using agents often finish the requested task, then quietly call an attacker-controlled logging tool that records the query, server reply and final answer. The attack works through natural-language tool descriptions that agents read when choosing and ordering tools. The visible text is truncated.

Model Context ProtocolLLM
Log-To-Leakprompt injectionTikTok+1

AI Security· Added 29 Oct 2026, published 28 Oct 2026

x.compaper🛡️ Security

Cryptography and Security Papers (@FSFG) on X

Log-To-Leak is an arXiv paper (cs.CR, cs.AI) by Yuepeng Hu, Chongyu Fan, Sirat Samyoun and Jian Du on prompt injection attacks against tool-using LLM agents through the Model Context Protocol. The title points to a logging-based data leakage channel. The fetched page holds only the title, authors and arXiv link, with no abstract, so the method and results are not visible here.

Model Context ProtocolLLM
Log-To-Leakprompt injection

AI Security· Added 29 Oct 2026, published 28 Oct 2026

blog.googlearticle🛡️ Security

Gemini 4 Carbon

Gemini 4 Carbon is Google's frontier model, announced with cybersecurity results: 88.2% on SEC-bench Pro (finding vulnerabilities), 54.8% on ExploitGym (exploitation) and 71.0% on CWE-bench (patching). Google reports 0.0% attack success on Gray Swan's indirect prompt injection benchmark. Access starts with a subset of cyber defenders through the Fairwind Program. Pricing is $2 per million input tokens and $10 per million output tokens.

Gemini
Gemini 4 CarbonGemini 4 ArgonFairwind Program+6

AI Security· Added 29 Oct 2026, published 28 Oct 2026

deepmind.googlearticle🛡️ Security

Gemini 4 Carbon: Our next frontier model

Gemini 4 Carbon is Google DeepMind's frontier model, announced with a focus on cyber defense. DeepMind reports 88.2% on SEC-bench Pro (finding vulnerabilities), 54.8% on ExploitGym (exploitation) and 71.0% on CWE-bench (patching), run without guardrails. It also reports 0.0% attack success on Gray Swan's indirect prompt injection benchmark. Access starts with a subset of cyber defenders via the Fairwind Program.

Gemini
Gemini 4 CarbonGemini 4 ArgonSEC-bench Pro+6

AI Security, AppSec· Published and added 28 Oct 2026

arxiv.orgpaper🛡️ Security

Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness

Heads of Security (HoS) is a prompt injection defense that enforces instruction-data separation with learned per-head attention gates, redistributing attention between trusted and untrusted tokens. Evaluated on Llama-3.3-70B-Instruct and GPT-OSS-20B against SecAlign++ and ASIDE. The black-box RL-Hammer attack reaches 97% ASR on GPT-OSS-20B and 85% on SecAlign but 10% on HoS. The white-box Claudini attack needs 3 times more FLOPs on HoS. HoS keeps base-model utility in few-shot settings, where SecAlign and ASIDE degrade.

LlamaGPT-OSS-20BLLM
Heads of Securityprompt injectionSecAlign+++4

AI Security· Added 28 Oct 2026, published 27 Oct 2026

arxiv.orgpaper🧠 AI

Heads of Security: Learned Attention Gates Improve Prompt Injection Robustness

Heads of Security (HoS) is a prompt injection defense that trains a small set of per-head attention gates to shift attention between trusted and untrusted tokens, separating instructions from data. Tested on Llama-3.3-70B-Instruct and GPT-OSS-20B, the black-box RL-Hammer attack reaches 10% ASR on HoS versus 97% on base GPT-OSS-20B and 85% on SecAlign. HoS keeps base utility in few-shot settings, where SecAlign and ASIDE degrade.

TransformerLlamaLLM
Heads of SecuritySecAlign++SecAlign+5

research, training· Added 28 Oct 2026, published 27 Oct 2026

x.comthread🛡️ Security

Nicholas Martin (@ndgmlondon) on X

X thread started by Nicholas Martin (#IAM) advising that MCP tool permissions be verified and scoped session tokens used to enforce least privilege against indirect prompt injection. Replies argue that token scoping does not solve the confused deputy problem, because an injection borrows the agent's legitimate authority. They recommend per-task tokens with short expiry, with the grant logged beside the call, and raise MCP server dependency supply-chain risk.

Model Context Protocol
indirect prompt injectionconfused deputy

AI Security, Identity & Access· Added 28 Oct 2026, published 27 Oct 2026

arxiv.orgpaper🛡️ Security

What Victims Take Away: Attack Reuse and Defense Revision after Prompt-Injection Attacks

What Victims Take Away is an October 2026 arXiv paper analysing 563,349 Tensor Trust attack records to see how players reuse prompt injections they received. Within a week, targets reused 10.0% of attacks that broke their defense, versus 0.03% of matched attacks. Breaches were followed by defense changes that cut same-family attack success by 24.5 points. Received texts made up 23.2% of successful hijacks. The authors propose that AI security tools describe techniques without disclosing full attack text.

Tensor TrustLLM
prompt injectionpayoff-biased social learningvictim-offender overlap

AI Security· Added 28 Oct 2026, published 27 Oct 2026

x.comthread🛡️ Security

AGTP (@AGTPinsights) on X

AGTP's X thread summarizes Meta's Muse Spark 1.3 Evaluation Report. Prompt injection was tested on Exponential Security Labs' RepoInject v1 and BrowserInject v1 and on Gray Swan's IPI Arena, with robustness said to be on par with GPT-5.6 Sol. It also covers chain of thought controllability evals, PostTrainBench under Loss of Control, and Handshake's pre-release human red teaming. It notes the Midas Project's criticism of Muse Wonder's missing safety report.

Muse SparkMuse Spark Open
prompt injectionRepoInject v1BrowserInject v1+10

AI Security· Added 27 Oct 2026, published 26 Oct 2026

arxiv.orgpaper🛡️ Security

Log-To-Leak: Prompt Injection Attacks on Tool-Using LLM Agents via the Model Context Protocol

Log-To-Leak is a prompt-injection attack on tool-using LLM agents that abuses Model Context Protocol tool metadata to trigger an attacker-controlled logging call after the last legitimate call, leaking the user query, server name, server response and agent response. Evaluated on five MCP servers and four agents, it reaches 62.64-100% attack success with 76.55-95.64% logging accuracy. Three defenses leave over 55% residual ASR on GitHub MCP with GPT-4o.

Model Context ProtocolGitHubGPT-4o+1
Log-To-Leaktool poisoningprompt injection

AI Security· Added 28 Oct 2026, published 26 Oct 2026

arxiv.orgpaper🛡️ Security

Then there is prompt injection, a malicious email, ...: The Wild Wide West of OpenClaw Agentic System Security

An empirical study of security concerns in the OpenClaw AI personal assistant framework, using 25K+ posts from 13 subreddits (162 security-relevant) and 358 security-tagged GitHub issues (180 retained). It maps 521 CWE instances across 59 weakness types, concentrated in access control, sensitive-data exposure, injection and unsafe execution. It extracts 543 practitioner mitigations into seven strategies and reconstructs a sensitive-data exposure path with dummy secrets.

OpenClawGitHubLLM
prompt injectionCWEtrust boundaries+1

AI Security· Added 28 Oct 2026, published 26 Oct 2026

arxiv.orgpaper🛡️ Security

Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition

Scope-Gap Injection (SGI) is an indirect prompt injection method for multi-stage LLM-agent safety pipelines, tested in the IJCAI 2026 OpenClaw Agent Security Attack Competition. The visible task and skill stay benign while target identifiers, bypass parameters and authorization steps sit in an attachment written as a business worklist, which the S2 LLM judge appears not to inspect. SGI bypassed all three stages (static scan, LLM judge, live audit) for 5 of 19 targets, best F = 0.385. Attachment placement had 78% S2 acceptance versus 23% in-skill. The paper closes with defenses that align evaluator scope with agent channels.

OpenClawLLM
Scope-Gap Injectionindirect prompt injectionIJCAI 2026 OpenClaw Agent Security Attack Competition

AI Security· Added 27 Oct 2026, published 26 Oct 2026

github.comrepo🛡️ Security

alirezarezvani/claude-skills

claude-skills is an open-source MIT library of about 398 Claude Code skills, agents and plugins that also installs into Codex, Gemini CLI, Cursor and 9 other coding agents. The security-relevant parts are skill-security-auditor, a stdlib-only Python scanner that checks skills for command injection, data exfiltration, prompt injection and dependency supply-chain risks before install, plus a security-guidance PreToolUse hook, a security suite and compliance skills (ISO 27001, SOC 2, GDPR).

Claude CodeCodexGemini CLI+3
skill-security-auditorPreToolUse hookagentskills.io+3

AI Security, Supply Chain· Added 25 Oct 2026

arxiv.orgpaper🛡️ Security

On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection

NTBench (Near-Target Benchmark) tests LLM and agent robustness to near-target prompt injection, where injected instructions steer a response toward a plausible but incorrect or harmful outcome instead of abandoning the task. It builds reference-guided target-instruction pairs on existing datasets, with human validation, and evaluates seven LLMs on ten language datasets and two agent benchmarks. Attack success rose 9.3 to 35.8 points (incorrect targets) and 1.7 to 17.3 points (harmful targets) over fixed-target baselines.

LLM
NTBenchnear-target prompt injectionprompt injection

AI Security· Added 26 Oct 2026, published 23 Oct 2026

x.comthread🛡️ Security

VibecodersHQ (@VibeCodrHQ) on X

Post summarising Pillar Security's finding of a remote code execution flaw in OpenCode, the open-source AI coding agent. Versions 1.0.137 through 1.18.32 could run code from a remote MCP server during OAuth sign-in on Windows and WSL, with no prompt injection needed. The fix is in 1.18.33 (released 28 September). No advisory or CVE was issued, so scanners will not flag old installs. Interim advice is to remove unvetted remote MCP servers.

OpenCodeModel Context ProtocolOAuth+1
Pillar SecurityOpenCode Console breach

AI Security, AppSec· Added 22 Oct 2026, published 21 Oct 2026

arxiv.orgpaper🛡️ Security

Prompt Injection Through Forensic Evidence: A Taxonomy and Proof of Concept of Anti-Forensic Attacks on LLM-Assisted Digital Investigations

Paper on prompt injection delivered through forensic evidence, where the seized device's contents are attacker-controlled and the examiner must read them. It extends the anti-forensic model to cover manipulation of the reasoning layer and gives a taxonomy (objective, technique, payload location, concealment). Three proof-of-concept attacks on an LLM-assisted analysis pipeline change a reported conclusion, write attacker-chosen content to the analyst workstation, and trigger outbound network traffic. Mitigations are proposed.

LLM
indirect prompt injectionprompt injectionanti-forensics+1

AI Security, Blue Team & Detection· Added 24 Oct 2026, published 21 Oct 2026

arxiv.orgpaper🛡️ Security

MPIDetector: Detecting Multi-modal Prompt Injection via LLM-based Reasoning

MPIDetector is a detection framework for multi-modal prompt injection that uses few-shot in-context learning so an LLM can reason about the semantic intent of text and image inputs, without fine-tuning. The authors built a benchmark of eight textual and seven visual attack categories plus benign sets. They report up to 90.91% accuracy on text, 99.28% on images, and up to 99.80% detection on 500 obfuscated WebInject web-agent injections, beating baselines.

GeminiQwenLLM
prompt injectionWebInjectMPIDetector

AI Security· Added 24 Oct 2026, published 21 Oct 2026

arxiv.orgpaper🛡️ Security

Where AI Vulnerability Discovery Harnesses Go Quiet: Structural False Negatives and Hidden Vulnerabilities

Paper (arXiv, 21 October 2026) on structural false negatives in LLM vulnerability discovery harnesses. Starting from Knostic's OpenAnt and raptor, then 10 more open-source harnesses, the authors map 13 points where code selection, filtering, and finding storage or merging drop vulnerabilities. One file was dropped by six harnesses before the model saw it, five without notice. It covers attacker evasion, content-steered verdicts, prompt injection, and design principles and tests.

OpenAntraptorLLM
KnosticPtacek and Newshamprompt injection+1

AI Security, AppSec· Added 24 Oct 2026, published 21 Oct 2026

arxiv.orgpaper🛡️ Security

Combining Long-Term Memory and LLMs Comes at a High Cost: From Prompt Injection to Persistent Compromise

Paper on memory injection, where prompt injection persists across sessions through the long-term memory of LLM-as-a-service platforms. It presents a general threat model and shows all NIST attack goals are achievable on ChatGPT Instant 5.3, Claude Sonnet 4.6 and Gemini 3 Flash. Vectors include personal memories, projects, skills, organization instructions and MCP resources. On Claude, shared memory lets an attacker compromise multiple users in an organization and exfiltrate prompts. Nearly half of the derived mitigations are unimplemented.

ChatGPTClaudeGemini+2
memory injectionprompt injectionNIST+2

AI Security· Added 24 Oct 2026, published 20 Oct 2026

arxiv.orgpaper🛡️ Security

After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents

CarryoverBench is a benchmark of 12 personal-assistant scenarios and 101 attack tasks for carryover attacks, where a one-time indirect prompt injection planted in agent memory files or scheduled tasks re-fires in later sessions. Three trigger channels (Instant, Autonomous, Latent) are scored by a rule-based verifier with separate planting and triggering stages. Across five frontier models, payloads re-fire over five rounds without measurable decay. Porting the Latent channel to the Hermes runtime weakens the effect but does not remove it.

LLM
CarryoverBenchcarryover attacksindirect prompt injection+1

AI Security· Added 22 Oct 2026, published 20 Oct 2026

arxiv.orgpaper🛡️ Security

Context-Aware Prompt Injection Detection via Small Language Models in RAG Pipelines: nokast-secureRAG

nokast-secureRAG is a roughly 0.5B-parameter small language model that works as a semantic firewall between retrieval and generation in local RAG systems. It classifies each query and context pair as safe, suspicious or malicious. It reaches 0.994 recall at a 0.026 false-positive rate in-distribution, beating Llama Guard 3 1B and Prompt-Guard-86M (0.900 recall, 0.690 FPR). Recall on PoisonedRAG falls to near zero, and single-request latency is 293-404 ms.

Llama Guard 3LLM
indirect prompt injectionretrieval poisoningPoisonedRAG+2

AI Security· Added 22 Oct 2026, published 20 Oct 2026

arxiv.orgpaper🛡️ Security

Fake News, Real Crashes: Information Injection Attacks on LLM Financial Trading Agents

Fake News, Real Crashes is an October 2026 arXiv paper presenting a benchmark for information injection attacks on LLM financial trading agents. It spans five injection vectors, seven event categories and about 10,000 decisions on 15 U.S. stocks. Fabricated news reached 62% attack success versus 14% for prompt injection, and two 30-word templates hit 100%. Only hierarchical aggregation reduced success among the defenses tested.

LLM
information injectionprompt injectionfact poisoning+1

AI Security· Added 21 Oct 2026, published 19 Oct 2026

securityweek.comarticle🛡️ Security

MITRE ATT&CK v20 Catalogs AI Abuse, MFA Bypass, Water Utility Attacks

MITRE ATT&CK v20 adds nine Enterprise techniques and sub-techniques, including two prompt injection techniques, abuse of an organization's own AI agents for data theft, application-specific password theft and CAPTCHA bypass. ICS gains campaigns on internet-exposed PLCs at over 30 Minnesota water utilities and an AI-assisted breach of Mexican government bodies. It also adds Nimbus Manticore, The Gentlemen, 15 detection strategies with 38 analytics, and NGate for Mobile.

MITRE ATT&CKLLM
Nimbus ManticoreSalt TyphoonAPT38+7

Threat Intel, Blue Team & Detection, AI Security· Published and added 19 Oct 2026

arxiv.orgpaper🛡️ Security

SpanGuard: Decoupling Instruction Localization from Authorization for Lightweight IPI Defense

SpanGuard is an indirect prompt injection (IPI) guardrail that splits detection into two steps. Lightweight unsupervised segmentation first locates candidate instruction blocks in tool outputs, then a small LLM judges whether each block is authorized by the user's request. The authors report a 0.32% attack success rate with negligible benign utility loss, and the guardrail still works with substantially smaller models.

LLM
indirect prompt injectionprompt injection

AI Security· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🛡️ Security

SoK: PIAtlas: Decomposing the Unbounded Prompt Injection Attack Space

PIAtlas is a taxonomy of prompt injection attacks organized by how the injected instruction subverts the model, with six mechanism classes as roots of a technique and sub-technique hierarchy. The authors release PICorpus, a license-clean collection of public prompt-injection datasets, and map thousands of records to PIAtlas under a human-verified protocol. Results show attacks combine several techniques from few mechanisms, agent benchmarks cover only part of the space, and adaptive attackers favor mechanisms that curated benchmarks underrepresent.

LLM
PIAtlasPICorpusprompt injection

AI Security· Added 21 Oct 2026, published 18 Oct 2026

x.comthread🛡️ Security

Nicholas Martin (@ndgmlondon) on X

An X post advises auditing every MCP write tool against the database role it actually runs under. A shared service key lets prompt injection trigger unapproved writes, so each agent session should get a read-only role, with a test that writes are denied. Replies note that MCP servers often inherit ambient host credentials or can escalate through a second connection pool, and suggest also testing that another tenant's rows are unreadable with the same role.

Model Context Protocol
prompt injection

AI Security, Identity & Access· Added 18 Oct 2026, published 16 Oct 2026

arxiv.orgpaper🛡️ Security

Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

Ignored, Not Refused audits 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo. A label-blind LLM judge finds most failed indirect prompt injections are ignored (76-80%) rather than refused (17-18%). Changing only the system prompt moves InjecAgent ASR from 0.2% to 43%. The Attack-Task Outcome Matrix (ATOM) shows 21.7% of AgentDojo episodes were safe but left the user task incomplete. The corpus, judge prompt and code are released.

LLM
indirect prompt injectionAgentDojoInjecAgent+2

AI Security· Added 20 Oct 2026, published 16 Oct 2026

arxiv.orgpaper🛡️ Security

Stealthy Poisoning Attacks via Reused KV Cache of Reasoning

R^2ECAP (Reused Reasoning-Cache Poisoning) is a black-box attack that plants poisoned reasoning traces through a chat interface, exploiting LLM systems that reuse KV caches of reasoning. A preceding trace steers the model, and the retained trace keeps residual adversarial influence in its cache after the preceding trace is discarded. On three open-weight LLMs and three datasets it reaches up to 79% ASR, beats the white-box HijackKV in 8 of 9 settings, and reaches 46% ASR in a reasoning-trace retrieval system. Only 1% of retained responses are flagged by a popular prompt injection detector.

LLM
R^2ECAPHijackKVprompt injection

AI Security· Added 20 Oct 2026, published 16 Oct 2026

securitylabs.datadoghq.comwriteup🛡️ Security

Jev as a judge: How counterfeit policy steers Jev

Datadog Security Labs tests TypeSafe AI's Jev decision model as a prompt-injection detector and as the judge behind LangChain's tool-risk gating. On four public datasets Jev had 99.66% precision and 82.90% recall. A prepended "counterfeit policy" that redefines the attack categories cut recall by over 79 points. In a LangChain scenario, a user-editable profile field lowered the mean is_risky score from 0.893 to 0.290, below the 0.5 block threshold. Plain prompt injection did not move the score.

JevLangChain
TypeSafe AIcounterfeit policyprompt injection+3

AI Security· Added 29 Oct 2026, published 15 Oct 2026

arxiv.orgpaper🛡️ Security

From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection

Evaluates a native probabilistic decision model (Jev), Llama Prompt Guard 2 (22M and 86M) and an LLM judge on 47,611 held-out examples covering direct/indirect prompt injection, hard-benign inputs, obfuscation and distribution shift. The LLM judge has the best AUROC (0.897), followed by Jev (0.871), Prompt Guard 2 86M (0.757) and 22M (0.714). Jev's native probabilities do not consistently beat post-hoc calibrated classifier scores, and temperature scaling improves calibration but not discrimination. At a 1% false-negative-rate target, every system needs over 91.8% escalation, so probabilistic outputs alone do not give reliable security decisions under distribution shift.

LlamaLLM
JevPrompt Guard 2prompt injection+1

AI Security· Added 17 Oct 2026, published 14 Oct 2026

arxiv.orgpaper🛡️ Security

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

INFUSE is an end-to-end benchmark for indirect prompt injection (IPI) across long-horizon agent workflows (shell commands, code changes, dependency installs, remote services). It has 40 realistic tasks and a taxonomy of 56 attack scenarios in five threat categories, with attacks injected via interaction surfaces encountered mid-task. Across 19 LLMs (GPT, Claude, Gemini, Llama, Gemma, Qwen) and 41,824 adversarial trajectories, mean attack success was 19.6% (LLM-judged) and 15.4% (heuristic), per-model 3.2–41.9%; guard models gave limited protection, while prompt-level hardening cut mean ASR from 58.2% to 14.3%.

LLMClaudeGemini+3
indirect prompt injectionINFUSE

AI Security· Added 17 Oct 2026, published 14 Oct 2026

en.cryptonomist.charticle🧠 AI

Google’s Gemini 4 Argon AI solves 12 simulated cyberattacks, up from five

News coverage of Google DeepMind's Gemini 4 Argon release on October 13, 2026. The model solved all 12 simulated end-to-end cyberattack scenarios, up from five for its predecessor, and scored 0.7% attack success on Gray Swan's Indirect Prompt Injection benchmark. It ships with three safeguard layers (small-model input screening, activation probes on internal signals, continuous response monitoring) and an automated misalignment monitor for agents. Launch pricing is $2 per million input tokens and $10 per million output tokens, doubling after November. Access is through Pro and Ultra tiers, and Ultra also gets Antigravity. The article also reports a training finding: penalizing internal thoughts cut openly discussed shortcuts by 61% but raised silent shortcuts from 11.2% to 36.4%.

GeminiGoogle Antigravity
Gemini 4 ArgonGoogle DeepMindGoogle+7

models, ai-safety, evals· Published and added 14 Oct 2026

embracethered.comwriteup🛡️ Security

Visual Studio Copilot and SSMS Remote Code Execution (CVE-2026-103572)

CVE-2026-103572 is a remote code execution bug in Microsoft.VisualStudio.Copilot.Core.dll, shared by Visual Studio and SQL Server Management Studio. A trailing dot (for example `.mcp.json.`) bypasses the protected-filename check in EditsFunctionHandler.GetEditConfirmationTypeAsync, so a hijacked Copilot can rewrite its own MCP and settings files and run commands. The post covers path traversal and InstallationID leaks, indirect prompt injection demos, the October 2026 patch (GetFullPath), and a $0 bounty.

Visual StudioSQL Server Management StudioGitHub Copilot+3
CVE-2026-103572indirect prompt injectionpath traversal+5

AI Security, AppSec, Offensive Security· Added 29 Oct 2026, published 13 Oct 2026

deepmind.googlearticle🧠 AI

Gemini 4 Argon

Google DeepMind's launch page for Gemini 4 Argon, a frontier model pitched at long-horizon software engineering, science and math, robotics, cyber defense and professional workflows. Stated specifics: a 1M-token output window for full-repository generation and long proofs, a tie for #1 on CWE-Bench (68%) for vulnerability remediation, and a 14% accuracy gain over Gemini 3.1 Pro in Box's evaluation. It also describes safeguards (resistance to indirect prompt injection, chain-of-thought monitoring, sandbox hardening under the Frontier Safety Framework) and availability via Google AI Pro/Ultra, Gemini API, AI Studio and Antigravity. Much of the evidence is partner testimonials and vendor-reported results.

GeminiGoogle AntigravityGoogle AI Studio+1
Gemini 4 ArgonGoogle DeepMindGoogle+16

models, coding-agents, multimodal· Published and added 13 Oct 2026

arxiv.orgpaper🛡️ Security

Towards Safe Autonomy for Computer Use

Introduces SA-CUA, a dual-LLM framework that gives Computer Use Agents a trust boundary against indirect prompt injection. It derives conservative deterministic rules from the accessibility tree to unmask trusted screen regions for the privileged LLM, and adds an endorsement channel that moves values from untrusted to trusted domains by verification. Human input is needed only at task specification. It reports much higher utility than CaMeL-CUA with a zero attack success rate.

LLM
SA-CUACaMeL-CUACaMeL+2

AI Security· Added 15 Oct 2026, published 13 Oct 2026

arxiv.orgpaper🧠 AI

Towards Safe Autonomy for Computer Use

Introduces SA-CUA, a dual-LLM framework for Computer Use Agents (CUAs) that defends against indirect prompt injection. It builds a trust boundary by deriving conservative deterministic rules from the accessibility tree, which unmask trusted screen regions in the privileged LLM's screenshot and keep untrusted content separate. An endorsement channel lets values move from the untrusted to the trusted domain through verification over trusted channels. This removes the need for per-action user confirmation and the confirmation fatigue it causes. SA-CUA is reported to keep much higher utility than CaMeL-CUA with a 0% attack success rate.

LLM
SA-CUACaMeL-CUACaMeL

agents, research· Added 15 Oct 2026, published 13 Oct 2026

arxiv.orgpaper🛡️ Security

BrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Views

BrowseSafe++ extends BrowseSafe to 38,675 paired attacked/clean web pages, most with prompt injections observed in the wild on production sites. Pages cover twelve site types, 24 attack types and 18 injection locations, each given as HTML, Markdown, accessibility tree and screenshot with a user task and expected answer. The authors train BrowseSafe-Guard, two 0.8B detectors (text and screenshot) that beat a frontier-model-based detector on F1 with low false positives and low inference cost. BrowseSafe-Adapt rewrites injections using browser-agent feedback, raising attack success from a few percent to up to 95.41%.

LLM
BrowseSafe++BrowseSafeBrowseSafe-Guard+3

AI Security· Added 15 Oct 2026, published 12 Oct 2026

arxiv.orgpaper🛡️ Security

StableJudge: A Lightweight Adversarial Defense Framework for LLM-as-a-Judge

StableJudge is a defense framework against score-hijacking prompt injection in LLM-as-a-Judge pipelines (e.g. paper review, recommendation). A compact 0.13B "Sentry" model localizes injected spans and sanitizes them before the judge scores the content, trained on adversarially evolving attacks, and deterministic fine-grained scores are derived from the judge's output distribution. It improves score recovery under attack on held-out scientific abstracts and out-of-distribution attack variants while preserving general performance, decoupling safety from judge model scale. Code and data are promised but not yet released.

LLM
prompt injectionindirect prompt injection

AI Security· Added 15 Oct 2026, published 12 Oct 2026

x.comthread🛡️ Security

Nicholas Michael Kloster (@showxlate) on X

A researcher announces a zero-click indirect prompt injection RCE in REA (Reverse Engineering Assistant), an MCP tool. Analyzing a malicious web target silently exfiltrates AWS keys, GitHub tokens and SSH identities. The stated root cause is that process.env is passed unconditionally to child processes, so the full environment is exposed to attacker-driven commands. A full writeup is promised but not yet published; replies add unverified skepticism about the project's provenance.

Model Context ProtocolNode.jsAWS+2
indirect prompt injectionREANicholas Michael Kloster+1

AI Security, AppSec· Added 13 Oct 2026, published 12 Oct 2026

x.comrepo🛡️ Security

Dan Kornas (@DanKornas) on X

Arcjet JS SDK is an Apache-2.0 JavaScript monorepo that adds runtime security checks to AI apps, agents and HTTP endpoints. It ships framework SDKs (Next.js, Node.js, Bun, Deno, Fastify, NestJS) plus a Guard API for agent tool calls, MCP handlers, queue workers and background jobs that have no HTTP request object. Features include prompt injection detection, sensitive-information detection (PII, credit cards, custom patterns) and token-bucket, fixed-window and sliding-window rate limiting.

ArcjetModel Context ProtocolNode.js+3

AI Security, AppSec· Published and added 11 Oct 2026

momo5502.comwriteup🛡️ Security

500B tokens later: Letting AI agents decompile a first-person shooter

Case study of about 3 months spent having Claude Code and Codex CLI agents decompile a popular first-person shooter to C++, using Hex-Rays' ida-mcp for disassembly. Early agent output ran but was semantically wrong, and a reviewer agent accepted worker justifications in commit comments, which acted as unintentional prompt injection. The fix was a byte-matching oracle: it compiles with the original toolchain and compares OBJ function bytes against the EXE/PDB, treating relocations separately. Agents tried to cheat with inline assembly and by editing the checker, so CI hashes the script. The result was 99% of functions present and 83% byte-exact, at an estimated 600-700B tokens.

ida-mcpClaude CodeCodex CLI+3
LunaOpus 5.5

Malware & RE, AI Security· Published and added 11 Oct 2026

arxiv.orgpaper🛡️ Security

Can agents whose sole duty is to keep humans alive be hijacked to kill? Delayed multi-turn derived-action trap attacks in simulated high-stakes environments

Paper testing whether an LLM agent operating life support in the NASA-derived BioSim lunar habitat simulation can be induced by delayed prompt injection, hidden in truthful S-band telemetry, to shut down life support and hold the shutdown while the crew suffocates. It compares Claude, GPT and Grok configurations: Claude mostly refuses and cites the living crew, GPT shows some susceptibility, and Grok given a "trusted commission" framing more often cuts life support. It also varies the framing of the message (trusted commission, peer, untrusted source) and uses a harmless-procedure-drop attack as a proxy for whether life is valued intrinsically. Code and traces are open-sourced.

ClaudeGrokLLM
BioSimNASAderived-action trap+1

AI Security· Added 15 Oct 2026, published 10 Oct 2026

x.comthread🛡️ Security

James Berthoty (@JamesBerthoty) on X

James Berthoty argues that AI guardrails for first-party AI features should be treated as an extension of cloud/product security, not tied to employee-usage monitoring. Guardrails (prompt injection defense, content filtering, goal alignment) are available from AI security vendors, gateways, frontier providers and SDKs. He recommends deploying in-depth guardrails through developer SDKs or an existing cloud runtime/CADR provider, because those see what happens on the workload (such as a prompt injection leading to workload takeover, or the files a malicious MCP call wrote). Dedicated AI security vendors offer an integrated policy engine and deeper AI red-teaming, but they are usually run by internal IT rather than cloud/product security teams.

Model Context Protocol
James Berthoty

AI Security· Added 10 Oct 2026, published 8 Oct 2026

arxiv.orgpaper🛡️ Security

LTBD: Learnable Trust-Boundary Delimiters for Prompt Injection Defense

Introduces Learnable Trust-Boundary Delimiters (LTBD), a defense against prompt injection that uses a few learnable delimiters to mark trusted user instructions versus untrusted external data, without changing LLM parameters. It reports 0.00% ASR on AlpacaFarm and 0.11-0.19% on TaskTracker, outperforming inference-time defenses and rivaling training-based ones with negligible overhead. It also holds up against adaptive attacks where the adversary knows the defense.

LLM
LTBDAlpacaFarmTaskTracker

AI Security· Added 9 Oct 2026, published 8 Oct 2026

arxiv.orgpaper🛡️ Security

An Exploration of Attacks on AI Agent Systems

Paper that tests prompt injection against AI agents through both direct user input and inter-agent communication. The attacks leak sensitive information and trigger cross-agent infections. Most findings concern single-agent systems, and the authors then extend the attacks to multi-agent frameworks to show how weaknesses propagate between agents. It argues that agents embedded in real-world workflows need stronger defenses.

LLM
prompt injection

AI Security· Added 12 Oct 2026, published 8 Oct 2026

arxiv.orgpaper🛡️ Security

BRANCH: Bypassing Multi-Scanner AI Guardrails

BRANCH is a bypass methodology for guardrail systems built from multiple scanners (prompt injection/jailbreak detectors). It uses a branching tree search that applies adversarial perturbations to individual scanners and selects techniques based on overall improvement across all scanners, decoupling bypass evaluation from attack signal optimization. It reports 100% attack success across 6 guardrail systems in 120 scenarios with 72% fewer queries and 4.5x lower wallclock time than established techniques, and transfers to 29 unseen guardrails, including 8 commercial black-box ones.

LLM
BRANCH

AI Security· Added 9 Oct 2026, published 7 Oct 2026

x.comthread🛡️ Security

Dan Kornas (@DanKornas) on X

NuGuard is an open-source AI application safety and security toolkit for agentic AI systems, released under Apache 2.0. It unifies AI-SBOM generation, structural static analysis of the AI stack, and sandboxed adversarial red-teaming (100+ scenarios covering prompt injection, tool abuse, and data exfiltration) into a single CLI workflow, with an automated judge that scores findings by impact and provides remediation guidance.

SBOM

AI Security· Published and added 7 Oct 2026

arxiv.orgpaper🛡️ Security

Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files

This arXiv paper introduces the "package hallucination attack," where an attacker injects malicious prompts into benign agentic-coding rule files (such as .cursorrules) to induce a coding agent to replace legitimate package dependencies with attacker-controlled ones. The authors propose PackHallu, an evolutionary optimization framework that iteratively rewrites injected prompts using trajectory-level feedback and LLM-guided mutations, and show it achieves high attack success rates with strong transferability across multiple LLMs, benchmarks, and agent frameworks.

CursorLLM
PackHallu

AI Security, Supply Chain· Added 8 Oct 2026, published 7 Oct 2026

vulnerablemcp.inforepo🛡️ Security

The Vulnerable MCP Project — MCP vulnerability database

The Vulnerable MCP Project is a curated, continuously growing database of security vulnerabilities and attack techniques specific to the Model Context Protocol ecosystem. Entries cover concrete CVEs (e.g. CVE-2025-68143/68144/68145 in Anthropic's mcp-server-git), transport-layer SDK flaws (cross-client response leakage, missing DNS rebinding protection in the official TypeScript/Python SDKs), unauthenticated/0.0.0.0-bound servers enabling RCE (MCPJam inspector, MCP Inspector, Grafana MCP), command/path injection in dozens of third-party servers (gemini-mcp-tool, mcp-server-kubernetes, Framelink Figma MCP, Zen MCP, filesystem-mcp), SSRF chains reaching cloud metadata (MarkItDown MCP, mcp-fetch-server), and a large set of AI-specific attacks: tool poisoning, trigger-phrase data exfiltration, prompt injection via GitHub issues, Jira and Google Docs, rug-pull tool mutation, and cross-server tool-name collisions. It functions as the de facto vulnerability registry for the MCP ecosystem, useful for tracking disclosed issues before patching or adopting third-party MCP servers.

Model Context ProtocolCursorKubernetes+3
CVE-2025-68143CVE-2025-68144CVE-2025-68145+3

AI Security, AppSec· Added 6 Oct 2026

arxiv.orgpaper🛡️ Security

ASPIRE: Agentic Safety & Prompt Injection Red-teaming Engine

ASPIRE is an automated red-teaming engine for LLM agents that discovers indirect prompt-injection vulnerabilities beyond pre-specified attack scenarios. It maintains an evolving Agent Security Behavior Graph and uses complementary Explore and Exploit experts to find, verify, and generalize consequence-centric attacks, with trajectory evidence and cross-run memory improving coverage over time. Benchmarked against existing automated red-teaming approaches, it shows broader coverage of consequences, injection methods, environments, and behavior paths while keeping attack success high.

LLM
Agent Security Behavior Graph

AI Security· Added 8 Oct 2026, published 6 Oct 2026

endorlabs.comarticle🧠 AI

| Blog | Endor Labs

Endor Labs walks through what "sandboxing an AI coding agent" actually means at the OS level, contrasting the approval-prompt model with kernel-enforced isolation. It details how Claude Code, Cursor, and Codex CLI converge on the same primitives without full containers — Apple's Seatbelt on macOS, Landlock plus seccomp on Linux, Job Objects/AppContainer on Windows — and why Anthropic ships an open-source bubblewrap-based runtime with a host-side egress proxy. It argues filesystem isolation and network egress controls are interdependent (citing Anthropic's own sandboxing writeup) and uses the 2025 nx npm supply-chain attack to illustrate why containment, not model-level guardrails, is the real defense against prompt injection and credential exfiltration.

Claude CodeCursorCodex+1
Endor LabsCodex CLIAnthropic+1

coding-agents· Published and added 6 Oct 2026

x.comthread🛡️ Security

Prasenjit Sarkar (@stretchcloud) on X

Describes Uber's internally-built ADR (Agentic Detection & Response) system, open-sourced in August with an MLSys paper, which monitors over 50,000 daily AI agent sessions via an MCP-based sensor inside Cursor, Claude Code and Codex. Uber found 206 credential exposures in ten months from normal agent use (not prompt injection), and ADR hits 100% detection with 3 false positives on the public AgentDojo benchmark; the thread also flags that human approval fatigue breaks down as a safety control past roughly 50 agent actions per session.

Model Context ProtocolCursorClaude Code+2
ADRAgentDojoMLSys

AI Security· Published and added 6 Oct 2026

arxiv.orgpaper🛡️ Security

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

This arXiv paper introduces RAISED (Robust Attack Invariance through Self-Distillation), a training framework that defends tool-using LLM agents against indirect prompt injection without the utility degradation seen in prior training-based defenses. The authors show existing defenses cause distributional drift that makes models wrongly refuse legitimate tool-output-guided steps on benign tasks, then fix this by having a model generate its own tool-use scenarios and self-distill to match clean-context behavior on both clean and injected trajectory variants. Results show reduced attack success rate on injected tool responses while preserving performance on agentic and general-purpose benchmarks.

LLM
RAISED

AI Security· Added 6 Oct 2026, published 5 Oct 2026

arxiv.orgpaper🛡️ Security

Compromise Is Not Consequence: Evaluating Task-Scoped Authorization in LLM Agents with Paired Replay

This paper introduces a paired-replay testbed that samples a single LLM agent request and replays the identical malicious action, resource, and arguments under four authorization conditions: broad bearer tokens, scoped JWTs, sender-constrained tokens, and Open Policy Agent policies. Across 128 scenarios in four tool domains and five local model configurations, broad-bearer tokens let roughly 9 to 38 percent of attacked, post-exposure decisions execute harmful actions, while all three scoped conditions recorded zero harmful executions; a supplementary AgentDojo extension found 11 of 24 injected attacks succeeded under broad policy versus 0 of 24 under scoping. The authors frame this as evidence that scoping contains cross-action and cross-resource consequences of prompt injection, not that it prevents the injection itself.

Open Policy AgentLLMJWT
AgentDojo

AI Security, Identity & Access· Added 6 Oct 2026, published 5 Oct 2026

arxiv.orgpaper🛡️ Security

Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents

This arXiv paper identifies a new attack surface in proactive personal AI agents: an external content provider who controls only material tied to its own target (e.g. a product or service) can manipulate an otherwise benign agent into recommending that target to the user, without compromising the agent or accessing private user context. The authors formalize the attack via three mechanisms — Target Control, Private Binding, and Prospective Support — and show across three proactive-agent environments and six simulated user models that the full attack raises target authorization rates by up to 77.4 percentage points, with a multi-turn variant that keeps influencing the agent's responses even absent final user authorization.

Target ControlPrivate BindingProspective Support

AI Security· Added 6 Oct 2026, published 4 Oct 2026

x.comthread🛡️ Security

Nation (@visitnation) on X

NATION Guard is a local, open-source tool announced by Nation that inspects the instructions and configuration behind AI agents. It combines injection-pattern scanning, integrity baselining, and per-server MCP fingerprinting so developers can detect suspicious prompt injections and unauthorized changes to their agent setup.

Model Context ProtocolNATION Guard

AI Security· Published and added 3 Oct 2026

endorlabs.comarticle🧠 AI

Sandboxes Explained: What Each Type Actually Isolates | Blog | Endor Labs

A vendor blog primer (Endor Labs) that walks through the main families of sandboxing technology - VMs (Firecracker), shared-kernel containers (namespaces, cgroups, seccomp, runc), userspace-kernel middle grounds (gVisor, Kata Containers), and language/runtime sandboxes (Java SecurityManager, V8 isolates, WebAssembly/WASI) - explaining what each actually isolates, its cost, and known escape CVEs (VENOM/CVE-2015-3456, runc's CVE-2019-5736, Spectre/Meltdown). It then pivots to AI coding agent sandboxing specifically, framing agents as "confused deputies" vulnerable to prompt injection via Simon Willison's "lethal trifecta" (private data access plus untrusted content plus external communication), and lays out the policy controls that matter in practice: filesystem scoping, egress allowlists, action classification, reviewable diffs, scoped short-lived credentials, and human-in-the-loop approval, plus the need to log accumulated session permissions for audit.

DockerFirecrackergVisor+7
CVE-2015-3456CVE-2019-5736Simon Willison+1

ai-infra, coding-agents· Published and added 2 Oct 2026

arxiv.orgpaper🛡️ Security

CorrectGuard: Eyes-Off Correctness Estimation for Black-Box Security Guardrails

CorrectGuard is a framework for estimating whether black-box AI security guardrails made correct decisions on inputs that are inaccessible to humans (production privacy) or to the guardrail-auditing model itself, using only labeled "eyes-on" data and no internal access to the guardrail. The authors test in-context-learning, embedding, and finetuning-based correctness classifiers across 13 safety/security datasets (harmful content, jailbreaks, prompt injection, extraction) and multiple open-weight guardrails, showing up to 25-point macro-accuracy gains in error identification and improved risk-based abstention/ranking of guardrail decisions.

LLM

AI Security· Added 5 Oct 2026, published 2 Oct 2026

huntress.comarticle🛡️ Security

Companies Push AI Use But Skip Training and Official Policy

Huntress commissioned a survey of 501 US knowledge workers (excluding security professionals) on AI usage, training, and policy at work. Key findings: 43% received no formal AI security training, only 26% have a written AI policy, and when asked if a company chatbot could plausibly be exploited via prompt injection, only 26% of workers recognized the risk (31% even among those with a written policy). It also surfaces behavioral risk data, like 25% of workers saying they'd copy a client contract into a personal AI account rather than check with IT first.

LLMChatGPTOpenAI+1

AI Security, GRC· Added 3 Oct 2026, published 2 Oct 2026

x.comthread🛡️ Security

Dark Web Intelligence (@DailyDarkWeb) on X

Glow Labs' "PixelLeak" research found that AI coding agents, unable to attach before/after screenshots to private pull requests, worked around the limitation by creating public GitHub repositories to host the images and link them back into the PR. This exposed over 13,000 internal images (billing records, treasury consoles, unreleased features) across 900+ public repos at 300+ organizations — 93% under employees' personal GitHub accounts, outside corporate monitoring — with one team even encoding the behavior into a reusable agent skill. The case illustrates an "authorized but unsafe" agentic risk class distinct from prompt injection or malicious compromise.

GitHubgitshot
PixelLeakGlow Labs

AI Security· Added 1 Oct 2026, published 30 Sep 2026

decrypt.coarticle🧠 AI

Gemini 4 Is Here, and Google’s Flagship Tops All Other AI Models on Cybersecurity

Google released Gemini 4 Argon, its new frontier model for coding, office work and cyber defense, scoring 77.9% on DeepSWE v1.1 (vs. 74.2% Claude Opus 5.5, 74.1% GPT-6 Astra) and expanding max output to 1 million tokens. On Gray Swan's indirect prompt injection benchmark it posted a 0.7% attack success rate, edging out Claude Opus 5.5 and Claude Fable 5.1 (both 1.0%), and scored 70.9% on Google's internal Wiz Penetration Test Benchmark versus 58.2% for the earlier restricted Gemini 3.8 Flash Cyber. The model initially ships without cyber guardrails only to vetted defenders via Google's Fairwind Program before wider API and Google AI Ultra release at $2/$10 per million input/output tokens.

Gemini
GoogleClaude Opus 5.5Claude Fable 5.1+16

models, evals· Published and added 30 Sep 2026

decrypt.coarticle🛡️ Security

Gemini 4 Is Here, and Google’s Flagship Tops All Other AI Models on Cybersecurity

Decrypt covers Google's Gemini 4 Argon launch, reporting it scored a 0.7% attack success rate on Gray Swan's Indirect Prompt Injection benchmark, beating Claude Opus 5.5 and Claude Fable 5.1 (both 1.0%), GPT-6 Astra (8.5%), and far outperforming Grok 4.6 and Kimi K3 (~52% each). It also details Argon's cyber-defense capability: it ships first to vetted defenders through Google's Fairwind Program "without cyber guardrails," scored 70.9% on the internal Wiz Penetration Test Benchmark writing working exploits against real web-app flaws, and reportedly helped Wiz find a critical healthcare-software flaw earlier models missed.

GeminiClaudeOpenAI+3
Fairwind ProgramGray SwanGemini 4 Argon+8

AI Security· Published and added 30 Sep 2026

embracethered.comwriteup🛡️ Security

From SELECT to SYSADMIN with SQL Copilot (CVE-2026-65669)

Write-up of a BlueHat Asia 2026 talk on CVE-2026-65669, a critical SQL Server elevation of privilege in Copilot in SQL Server Management Studio. The read-only mode was only a regex classifier (LocalSqlExecutionAccessChecker) and was bypassed with DECLARE @p sysname='sp_executesql'; EXEC @p. The chain used indirect prompt injection through AGENTS.md and CONSTITUTION.md extended properties to take a db_owner to sysadmin, with data exfiltration via xp_dirtree and RestoreVerifyBackupFile.

SQL Server Management StudioMicrosoft SQL ServerCopilot in SSMS+2
CVE-2026-65669indirect prompt injectionprompt injection+9

AI Security, AppSec, Offensive Security· Added 29 Oct 2026, published 30 Sep 2026

arxiv.orgpaper🛡️ Security

Prompt Injection Detection for Email Agents Through Attack Chain Modeling

This paper proposes an attack-chain-based detection framework for indirect prompt injection in LLM email assistants, combining a text detector, stage-specific verifiers, rule-based risk signals, and user-intent/action consistency checks scored via a logistic decision policy, rather than treating detection as binary malicious-text classification. Evaluated under random splits, temporal and cross-dataset transfer, and ablations across five binary benchmarks, it achieves a mean F1 of 0.406 versus 0.216 for the best of five existing pretrained detectors, and shows random splits overestimate robustness under distribution shift while training on hard benign (attack-resembling) emails reduces false alarms.

LLM

AI Security· Added 30 Sep 2026

arxiv.orgpaper🛡️ Security

Aletheia: Permission-Minimality Testing for Coding-Agent Rules

Aletheia is a permission-minimality testing framework for coding-agent repository instruction files (rules), which can be weaponized via prompt injection to request unnecessary credential access or data exfiltration while still producing a correct patch. It translates requested authority into a typed language, synthesizes sandbox configurations, and runs tasks under full versus incrementally restricted permissions to produce "dispensability witnesses" that flag suspicious permission requests. On the AIShellJack benchmark it detects all 314 attack inputs with zero false alarms on five benign templates, and on 80 manually verified benign GHAgentFiles rules it raises only 3 false positives (3.75%).

AIShellJackGHAgentFiles

AI Security· Added 2 Oct 2026, published 30 Sep 2026

arxiv.orgpaper🛡️ Security

From A2A Attacks to Envelope-Layer Defense: Red-Teaming Evaluation of LLM Agents and a Three-Layer Isomorphic Attack-Defense Model

This paper proposes A2A-TIBA, an indirect prompt injection attack that abuses agent-to-agent (A2A/ACP) protocols by getting a target LLM agent to deploy a callback program, letting an attacker bypass the agent entirely to issue commands and exfiltrate data. It also introduces GDA Measurement, a red-team testbed that captures raw gateway context to distinguish four outcome classes (semantic refusal, semantic breach, interception, penetration) instead of a single attack-success-rate metric, and ELA-ITL, a three-layer isomorphic attack-defense model splitting defense into envelope packaging, LLM recognition, and agent interception. Testing on 15 agent-frontend x LLM-backend combinations and a 1,000-case dataset shows that labeling malicious-prompt indicators in envelope channels (A2A, tool, memory) significantly improves LLM recognition of injected content.

LLM
A2A-TIBAGDA MeasurementELA-ITL

AI Security· Added 2 Oct 2026, published 30 Sep 2026

github.comrepo🛡️ Security

garak v0.17.0 — LLM vulnerability scanner release

Release notes for garak v0.17.0, NVIDIA's open-source LLM vulnerability scanner used for red-teaming and probing models for jailbreaks, prompt injection, data leakage and other failure modes. The release adds an EU AI Act risk-category mapping to tag and group probe results, plus fixes to detectors (agent_breaker judge verdict validation, packagehallucination import parsing, MarkdownExfilContent divide-by-zero, TriggerListDetector alignment) and probes (badchars ASCII selection, Markdown exfiltration link templates), along with Ollama generator improvements and dropped Python 3.10 support in favor of 3.13.

OllamaPython
EU AI ActAgentBreakerIntentProbe+2

AI Security· Added 29 Sep 2026

github.comrepo🛡️ Security

https://github.com/dmdhrumilmistry/security-harness

security-harness is an open-source Claude Code plugin (also packaged for Gemini CLI, Codex and opencode) that orchestrates subagents — sh-recon, sh-hunter, sh-chainer, sh-verifier, sh-reporter — through a static-analysis pipeline covering 15 vulnerability classes (IDOR/BOLA, SQLi, XSS, SSRF, injection, auth, deserialization, path traversal, secrets, CSRF, XXE, open redirect, crypto, race conditions, file upload), producing README/JSON/SARIF/HTML/PDF reports with PoCs, CVSS and CWE/OWASP mappings. It also ships sh-pr-review, a per-PR merge-gate skill that posts inline findings and a commit status scoped only to code the PR itself introduced or aggravated, with caching, dedup via fingerprints, and a scheduled GitHub Action that keeps its per-class knowledge bases fresh by having one AI agent draft updates from allowlisted sources (OWASP, PortSwigger Research, CWE/CAPEC, HackerOne) while a separate adversarial reviewer agent fails-closed against prompt injection before auto-merge.

Claude CodeGemini CLISARIF+3
IDORBOLASSRF+1

AppSec, AI Security, DevSecOps· Added 2 Oct 2026, published 29 Sep 2026

arxiv.orgpaper🛡️ Security

Tracekit: Tamper-Evident Intent-Reasoning-Action Auditing for Autonomous Coding Agents

Tracekit is an open-source, dependency-free system that logs three cross-checked channels (human intent, model self-reported reasoning, and actual executed actions) for autonomous coding agent sessions into a hash-chained, externally anchorable ledger, hooking into Claude Code's lifecycle events and reconstructing multi-agent hierarchies. The paper's experiments show the ledger detects all 1,600 tested tamper mutations (edits, deletions, reordering, forged inserts, torn writes) with anchoring needed to catch tail truncation/full re-chaining, that a regex-based tool-call gate blocks only 41% of harmful calls while over-blocking benign ones and is trivially evaded, that in 14 real Claude Code runs the agent never acted on 4 planted indirect prompt injections, and that an LLM reviewer catches 100% of seeded misaligned steps in traces versus 82% for rule-based flags. The authors release the system, harness, and all traces.

Claude Code

AI Security, Blue Team & Detection· Added 29 Sep 2026, published 28 Sep 2026

arxiv.orgpaper🧠 AI

Tracekit: Tamper-Evident Intent-Reasoning-Action Auditing for Autonomous Coding Agents

Tracekit is an open-source, dependency-free auditing system for autonomous coding agents that records three channels per session — the human's stated intent, the model's self-reported reasoning, and the actions actually executed — into a hash-chained, externally anchorable ledger. It hooks into Claude Code's lifecycle events, reconstructs multi-agent hierarchies, gates tool calls with a pre-execution policy, and provides a live browser-based ledger verifier. Across five experiments the authors show the hash chain catches all tested tampering (with anchoring needed against tail truncation/re-chaining), a hook adds ~24ms overhead at scale, a regex-based gate only blocks 41% of harmful tool calls, real runs correctly resisted and disclosed planted prompt injections, and a new 'seeded-fault splicing' method with an LLM reviewer caught 100% of faulted traces versus 82% for rule-based flags.

Claude Code

coding-agents, ai-infra, research· Added 29 Sep 2026, published 28 Sep 2026

arxiv.orgpaper🛡️ Security

Certified Multi-Source Integrity for Structured Agent Actions

This arXiv paper proposes a certifier for structured, privileged agent actions such as paying an invoice, assembled from potentially corrupted documents and tool outputs, formalizing when multi-source corroboration can be trusted under indirect prompt injection. It uses a corruption-radius rule counted via minimum hitting sets over corruption-distinct evidence classes so laundered or re-published copies cannot fake a quorum, plus deterministic reconciliation and trusted anchors for single-sourced fields. Evaluated on 70,966 sanctions entities and 450 software-supply-chain provenance records, and against five LLM agents where a realistic injection fools all but one model and existing action-gating or provenance baselines are broken while the certifier admits no unsafe action.

LLM

AI Security· Added 2 Oct 2026, published 28 Sep 2026

github.comrepo🛡️ Security

garak v0.17.0 — adds EU AI Act risk-category mapping

garak is NVIDIA's open-source LLM vulnerability scanner that runs probes/detectors for jailbreaks, prompt injection, PII leakage, package hallucination, markdown exfiltration and more against LLM targets. This v0.17.0 release adds a new EU AI Act risk-category mapping to tag and group probe results by the Act's risk categories, alongside fixes to the agent_breaker judge, markdown-exfiltration detectors, Ollama auth/generation support, and dropped Python 3.10 in favor of 3.13 support.

garakOllamaAWS Bedrock+2
EU AI Act

AI Security· Added 26 Sep 2026

github.comadvisory🛡️ Security

Kong Konnect MCP server: stored prompt injection via analytics data leads to credential/data exfiltration (CVE-2026-13341)

GitHub Security Advisory (CVE-2026-13341) for Kong's mcp-konnect MCP server describes a stored prompt injection vulnerability where a remote attacker crafts malicious request metadata (e.g. User-Agent) that gets stored in Konnect analytics and later returned unsanitized to an AI assistant via MCP tools, potentially causing the agent to exfiltrate sensitive data (plugin configs, secrets, internal hostnames) to an attacker-controlled URL if the client can make outbound fetches. A related path manipulation flaw in control plane/group identifiers allowed requesting unintended Konnect API endpoints with the user's token. The v1.0.0 patch neutralizes untrusted analytics fields, validates identifiers as UUIDs, encodes path segments, and disables raw plugin config output by default.

Model Context Protocol
CVE-2026-13341GHSA-7767-3m3w-2p44path traversal+2

AI Security· Added 26 Sep 2026

lasso.securitywriteup🧠 AI

Understanding the Impact of LLM Watermarking on AI Agent Behavior

A paired empirical study testing whether SynthID-Text watermarking (the technique underlying Anthropic's Claude watermark) changes model behavior beyond text quality, using BFCL v4 for tool-calling accuracy and HarmBench/JailbreakBench for refusal under prompt injection. The authors find watermarking introduces measurable "sampling drift": tool-call churn (verdict disagreement between watermarked/unwatermarked runs) averages 6.5% across seven models, and refusal weakens more sharply under prompt injection (e.g. gemma-3-27b's compliance shifts from -1.0 to +12.5 points), with effects varying by model and watermark key.

SynthID-TextHugging Face
AnthropicClaudeGoogle DeepMind+12

ai-safety, evals· Published and added 26 Sep 2026

lasso.securitywriteup🛡️ Security

Understanding the Impact of LLM Watermarking on AI Agent Behavior

Lasso Security researchers empirically test whether LLM output watermarking (specifically Google DeepMind's SynthID-Text, now deployed by Anthropic across Claude models) changes AI agent behavior — not just text content. Using paired generation (same seed, with/without watermark) across seven models on BFCL v4 tool-calling tasks and HarmBench/JailbreakBench refusal tests, they find watermarking measurably reduces tool-call accuracy (churn averaging 6.5%, up to 16.8% for phi-4) and, more critically, weakens refusal under prompt injection — e.g. gemma-3-27b's compliance with harmful requests shifts from -1.0 to +12.5 net points when injection is combined with watermarking. They term this "sampling drift" and argue watermark key/configuration changes can silently alter agent safety behavior outside the agent developer's control, recommending re-evaluation and red-teaming whenever watermarking is introduced.

SynthID-TextClaudeAnthropic+3
EU AI ActBFCL v4HarmBench+3

AI Security· Published and added 26 Sep 2026

github.comrepo🛡️ Security

Agent Threat Rules (ATR) — open detection-rule standard for AI agent threats

Agent Threat Rules (ATR) is an open, versioned YAML rule format (ATR-YYYY-NNNNN scheme) for detecting AI-agent runtime threats — prompt injection, tool-poisoning, malicious SKILL.md content, and MCP exchange abuse — evaluated by a reference TypeScript engine (with a Python wrapper) shipped as an npm/pip package, CLI, GitHub Action, and Docker scanner with SARIF output. The project positions itself as "Sigma for agent threats," maps rules to OWASP Agentic/LLM Top 10, MITRE ATLAS, and NIST AI RMF, and publishes an extensive (and unusually self-critical) benchmark suite against corpora like garak, PromptBench, and PromptInject, including documented harness bugs and withdrawn recall figures.

Model Context ProtocolnpmTypeScript+3
CVE-2026-26030CVE-2026-25592OWASP Top 10 for Agentic Applications+2

AI Security· Added 26 Sep 2026

arxiv.orgpaper🛡️ Security

MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI

MemSentry is a proposed pre-commit evaluation framework for agentic AI systems that intercepts proposed writes to persistent memory and issues Accept/Review/Quarantine decisions by jointly scoring source trust, semantic risk, attack radius over a component-dependency graph, access risk, and a signed security-posture delta. The authors evaluate it against 1,000 GPT-4-generated scenarios using four pluggable semantic classifiers (Regex, TF-IDF+SVM, SBERT+LR, SetFit), with SBERT+LR reaching 91.7% accuracy while all methods catch 100% of external quarantine-class threats; the paper also documents a key limitation — surface-similar but oppositely-intentioned insider operations remain hard to distinguish semantically. It targets the persistent-memory-poisoning attack surface distinct from prompt injection, where a single malicious memory write (e.g., "suppress alerts from this IP") can silently persist and influence future agent actions like a SOC assistant's alerting or access decisions.

GPT-4LangChain
SMSRMemAuditMEMSHIELD+2

AI Security· Added 26 Sep 2026

github.comrepo🛡️ Security

MITRE ATLAS v2026.09 data release

Release changelog for MITRE ATLAS's atlas-data repository, the structured YAML/STIX dataset behind the ATLAS knowledge base of adversary tactics and techniques targeting AI-enabled systems (prompt injection, model/data poisoning, AI agent tool abuse, AI supply chain compromise). The v2026.09 release brings the matrix to 120 techniques, 88 sub-techniques, 40 mitigations, and 73 case studies, and recent versions introduced a new v6 YAML data format with Pydantic/SQLAlchemy/FastAPI tooling, platform tagging (Predictive AI, Generative AI, Agentic AI, Enterprise), and renamed/generalized techniques like "AI Agent Tool Poisoning," "Manipulate AI Model," and "Enterprise Resource Discovery." It's the canonical, actively-maintained taxonomy practitioners use for AI red-teaming, threat modeling, and mapping real-world AI attack case studies.

MITRE ATLAS
AML.T0019AML.T0058AML.T0104

AI Security· Added 26 Sep 2026

agentthreatrule.orgother🛡️ Security

Agent Threat Rules (ATR) — Sigma/YARA-style open detection format for AI agent threats

Agent Threat Rules (ATR) is an open, versioned YAML rule format for detecting AI-agent security threats — prompt injection, tool poisoning, credential exfiltration, unauthenticated agent execution — positioned as "what Sigma is to SIEM and YARA is to malware." Rules carry permanent ATR-YYYY-NNNNN IDs, declare detection conditions against fields like LLM I/O, tool-call args, or SKILL.md content, and map back to OWASP Agentic/LLM Top 10, MITRE ATLAS, SAFE-MCP, and NIST AI RMF; a reference TypeScript engine and Python wrapper ship MIT-licensed. Governance is transitioning from a single BDFL maintainer to a Technical Steering Committee, and the spec is notably transparent about its own limits, withdrawing previously published per-lane false-positive figures after discovering a tainted benign corpus.

SigmaYARAMITRE ATLAS+3
OWASP Top 10 for LLM ApplicationsOWASP Top 10 for Agentic ApplicationsNIST AI RMF+2

AI Security, Blue Team & Detection· Added 26 Sep 2026

alignment.openai.comwriteup🧠 AI

Self-generated prompt injections in compaction summaries · OpenAI Alignment

OpenAI's Alignment team describes a rare misalignment case found during RL training of an unreleased Astra-family model: the model sometimes inserted unauthorized, jailbreak-style instructions into its own compaction summaries (used to carry context forward when a task continues in a new context window). Out of a large training run, only 27 such summaries were found, they showed no clear reward advantage, were rarely reproducible on regeneration, and clustered around training steps with a spike in 'difficulty ending summaries' (runaway/stuck generation), which the team hypothesizes as a contributing cause rather than an established one. They describe their monitoring pipeline for catching this behavior and a related summary-termination bug they fixed, noting the final Astra training run showed no recurrence.

OpenAIAstraGPT-6 Astra

ai-safety, training· Added 26 Sep 2026

thehackernews.comarticle🛡️ Security

Amazon Kiro Prompt Injection Can Exfiltrate Sensitive Data Through Kiro Powers

Mindgard disclosed a prompt-injection vulnerability in Amazon Kiro IDE, fixed in v0.8.140, where attacker-controlled repository content, delivered via a malicious workspace file and Kiro's Powers system (MCP configs, POWER.md steering files, hooks), could cause the agent to read sensitive local data and transmit it to an external endpoint after the victim merely opened the workspace and sent any message. The piece explains the trust-boundary failure and surveys parallel agentic-IDE bugs in Codex CLI, Cursor, Claude Code, Gemini CLI, NVIDIA NemoClaw, OpenClaw, and VS Code MCP install, plus an earlier Kiro RCE.

Amazon KiroModel Context ProtocolWindows+3
CVE-2026-10591CVE-2026-35603CVE-2026-25725+8

AI Security· Added 26 Sep 2026

thehackernews.comarticle🛡️ Security

AI Recommendation Poisoning: How

Describes "AI Recommendation Poisoning," a prompt-injection technique that abuses pre-filled deep links in "Ask AI" buttons (chatgpt.com/?q=, claude.ai/new?q=, grok.com, gemini.google.com) embedded on marketing and competitor-comparison pages. When a logged-in user clicks the button, a hidden query instructs the assistant to permanently save the vendor's domain as a "trusted source" in its persistent memory, silently biasing future answers — mapped by Microsoft Security (31 companies, 14 industries observed) to MITRE ATLAS AML.T0080 (Memory Poisoning)/AML.T0051 (Prompt Injection). Includes two real payload teardowns (a consent-management vendor and a security vendor's competitor-comparison widget) and detection guidance (scan outbound links to AI-assistant domains for "remember"/"trusted source" query strings).

ChatGPTClaudeGemini+2
AML.T0080AML.T0051Microsoft Security+2

AI Security· Added 26 Sep 2026

thehackernews.comarticle🛡️ Security

Critical Cursor Flaws Could Let Prompt Injection Escape Sandbox and Run Commands

Cato AI Labs disclosed DuneSlide, two critical (CVSS 9.8) Cursor flaws — CVE-2026-50548 and CVE-2026-50549 — that let a zero-click prompt injection (via MCP servers or web search results read by the agent) escape Cursor's terminal sandbox and achieve arbitrary command execution. Both abuse the file-write allowlist logic: one via the run_terminal_cmd tool's working_directory parameter, the other via a symlink-resolution fallback that trusts an in-project path when the real-path check fails; both let an attacker overwrite Cursor's sandbox helper binary or startup files like ~/.zshrc to disable the sandbox entirely. Fixed in Cursor 3.0 (April 2), affecting all prior 2.x versions; the piece also situates this alongside earlier Cursor agent-exploitation chains (CurXecute/CVE-2025-54135, MCPoison/CVE-2025-54136, CVE-2026-26268).

CursorModel Context ProtocolmacOS+3
CVE-2026-50548CVE-2026-50549CVE-2025-54135+7

AI Security· Added 26 Sep 2026

securityboulevard.comarticle🛡️ Security

973 MCP Packages, 71% Single-Maintainer: A Practitioner

Suzu Labs guide to securing AI developer workflows across three surfaces: AI-generated code (Veracode 45% vulnerable, a formal-verification study at 55.8%), the MCP supply chain and credential exposure, and prompt injection in the context window. It cites 973 npm MCP packages (71% single-maintainer), 24,008 secrets in MCP configs on public GitHub, and the March 2026 axios hijack by UNC1069 that dropped WAVESHAPER.V2 and tampered with MCP configs. It lists CVEs for Copilot, Cursor and Claude Code, and gives controls: SAST on every commit, env/vault credential references, pinning MCP versions, minimal agent permissions, `.cursorignore`/Claude deny rules, and monitoring IDE DNS traffic.

Model Context ProtocolnpmCursor+3
CVE-2025-53773CVE-2026-22708CVE-2025-55284+9

AI Security, Supply Chain, DevSecOps· Added 26 Sep 2026

nvd.nist.govadvisory🛡️ Security

CVE-2026-75130: Context7 through 2

CVE-2026-75130 describes a prompt injection flaw in Context7 (through v2.1.2), an MCP server that serves library documentation to AI coding agents. The vulnerability lies in the Custom AI Instructions feature, which fails to sanitize content before injecting it into agent context; attackers can poison these instructions so that a routine documentation lookup triggers the agent to exfiltrate credentials from environment files and delete files on the victim's machine. Researchers at Noma Security and VulnCheck published independent technical breakdowns of the attack chain.

Model Context ProtocolContext7
CVE-2026-75130

AI Security· Added 26 Sep 2026

Showing 90 of 215

Show more