52 results for “Direct prompt injection”

What you seeeverything

All types52 Papers19 Articles13 Write-ups7 Repos5 Advisories3 Videos3 Threads2

arxiv.orgpaper🧠 AI

SecPO: Principled Adversarial Training for Prompt Injection Security

SecPO is a DPO-variant loss from UC Berkeley for training LLMs against indirect prompt injection. It replaces the undefended-model reference on injected input with its output on the clean input, a one-line change, and pairs with a PAIR-based adversarial training pipeline. On Qwen3.6-27B, AgentDojo ASR@800 is 8.8% against 78.8% for Meta-SecAlign, with utility unchanged across nine benchmarks. Code, training set and weights are released.

DPOPAIRGCG+4

SecPOUC BerkeleySizhe Chen+5

training, research· Added 3 Nov 2026, published 30 Oct 2026

arxiv.orgpaper🛡️ Security

SEAL: Sealed Queries for Local Sanitization against Indirect Prompt Injection

SEAL is a defense against indirect prompt injection for local LLM agents. It learns a soft prompt and scoring readout offline, then uses a frozen task model to score and delete injected tokens from tool outputs, with queries built from trusted initial context. On AgentDojo, attack success falls from 34.6% to 0.1% and Joint rises from 51.4% to 77.5%, using about 127x fewer defense tokens than PISanitizer.

LLMAgentDojoAgentDyn+1

indirect prompt injection

AI Security· Added 31 Oct 2026, published 29 Oct 2026

arxiv.orgpaper🛡️ Security

Appreciation Teaches Unauthorized Execution: Self-Evolution Degrades the Robustness of Personal Agents to Indirect Prompt Injection

Paper studying whether self-evolution weakens personal agents' robustness to indirect prompt injection (IPI). A two-phase method has the user ask the agent to follow tasks in external content, then tests with a third-party injected task the user did not authorize. Against the non-evolving base agent, attack success rose 14.0 to 48.8 points under skill evolution and 3.0 to 39.3 points under reinforcement learning. Code is released.

indirect prompt injection

AI Security· Added 31 Oct 2026, published 29 Oct 2026

blog.googlearticle🛡️ Security

Gemini 4 Carbon

Gemini 4 Carbon is Google's frontier model, announced with cybersecurity results: 88.2% on SEC-bench Pro (finding vulnerabilities), 54.8% on ExploitGym (exploitation) and 71.0% on CWE-bench (patching). Google reports 0.0% attack success on Gray Swan's indirect prompt injection benchmark. Access starts with a subset of cyber defenders through the Fairwind Program. Pricing is $2 per million input tokens and $10 per million output tokens.

Gemini

Gemini 4 CarbonGemini 4 ArgonFairwind Program+6

AI Security· Added 29 Oct 2026, published 28 Oct 2026

deepmind.googlearticle🛡️ Security

Gemini 4 Carbon: Our next frontier model

Gemini 4 Carbon is Google DeepMind's frontier model, announced with a focus on cyber defense. DeepMind reports 88.2% on SEC-bench Pro (finding vulnerabilities), 54.8% on ExploitGym (exploitation) and 71.0% on CWE-bench (patching), run without guardrails. It also reports 0.0% attack success on Gray Swan's indirect prompt injection benchmark. Access starts with a subset of cyber defenders via the Fairwind Program.

Gemini

Gemini 4 CarbonGemini 4 ArgonSEC-bench Pro+6

AI Security, AppSec· Published and added 28 Oct 2026

x.comthread🛡️ Security

Nicholas Martin (@ndgmlondon) on X

X thread started by Nicholas Martin (#IAM) advising that MCP tool permissions be verified and scoped session tokens used to enforce least privilege against indirect prompt injection. Replies argue that token scoping does not solve the confused deputy problem, because an injection borrows the agent's legitimate authority. They recommend per-task tokens with short expiry, with the grant logged beside the call, and raise MCP server dependency supply-chain risk.

Model Context Protocol

indirect prompt injectionconfused deputy

AI Security, Identity & Access· Added 28 Oct 2026, published 27 Oct 2026

arxiv.orgpaper🛡️ Security

Scope-Gap Injection: Exploiting Evaluator Channel Blind Spots in the OpenClaw Agent Security Competition

Scope-Gap Injection (SGI) is an indirect prompt injection method for multi-stage LLM-agent safety pipelines, tested in the IJCAI 2026 OpenClaw Agent Security Attack Competition. The visible task and skill stay benign while target identifiers, bypass parameters and authorization steps sit in an attachment written as a business worklist, which the S2 LLM judge appears not to inspect. SGI bypassed all three stages (static scan, LLM judge, live audit) for 5 of 19 targets, best F = 0.385. Attachment placement had 78% S2 acceptance versus 23% in-skill. The paper closes with defenses that align evaluator scope with agent channels.

OpenClawLLM

Scope-Gap Injectionindirect prompt injectionIJCAI 2026 OpenClaw Agent Security Attack Competition

AI Security· Added 27 Oct 2026, published 26 Oct 2026

arxiv.orgpaper🛡️ Security

After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents

CarryoverBench is a benchmark of 12 personal-assistant scenarios and 101 attack tasks for carryover attacks, where a one-time indirect prompt injection planted in agent memory files or scheduled tasks re-fires in later sessions. Three trigger channels (Instant, Autonomous, Latent) are scored by a rule-based verifier with separate planting and triggering stages. Across five frontier models, payloads re-fire over five rounds without measurable decay. Porting the Latent channel to the Hermes runtime weakens the effect but does not remove it.

LLM

CarryoverBenchcarryover attacksindirect prompt injection+1

AI Security· Added 22 Oct 2026, published 20 Oct 2026

arxiv.orgpaper🛡️ Security

SpanGuard: Decoupling Instruction Localization from Authorization for Lightweight IPI Defense

SpanGuard is an indirect prompt injection (IPI) guardrail that splits detection into two steps. Lightweight unsupervised segmentation first locates candidate instruction blocks in tool outputs, then a small LLM judges whether each block is authorized by the user's request. The authors report a 0.32% attack success rate with negligible benign utility loss, and the guardrail still works with substantially smaller models.

LLM

indirect prompt injectionprompt injection

AI Security· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🛡️ Security

Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

Ignored, Not Refused audits 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo. A label-blind LLM judge finds most failed indirect prompt injections are ignored (76-80%) rather than refused (17-18%). Changing only the system prompt moves InjecAgent ASR from 0.2% to 43%. The Attack-Task Outcome Matrix (ATOM) shows 21.7% of AgentDojo episodes were safe but left the user task incomplete. The corpus, judge prompt and code are released.

LLM

indirect prompt injectionAgentDojoInjecAgent+2

AI Security· Added 20 Oct 2026, published 16 Oct 2026

arxiv.orgpaper🛡️ Security

From Detection to Decision: Calibration and Deferral for Prompt-Injection Detection

Evaluates a native probabilistic decision model (Jev), Llama Prompt Guard 2 (22M and 86M) and an LLM judge on 47,611 held-out examples covering direct/indirect prompt injection, hard-benign inputs, obfuscation and distribution shift. The LLM judge has the best AUROC (0.897), followed by Jev (0.871), Prompt Guard 2 86M (0.757) and 22M (0.714). Jev's native probabilities do not consistently beat post-hoc calibrated classifier scores, and temperature scaling improves calibration but not discrimination. At a 1% false-negative-rate target, every system needs over 91.8% escalation, so probabilistic outputs alone do not give reliable security decisions under distribution shift.

LlamaLLM

JevPrompt Guard 2prompt injection+1

AI Security· Added 17 Oct 2026, published 14 Oct 2026

arxiv.orgpaper🛡️ Security

INFUSE: A Unified Surface Benchmark for Indirect Prompt Injection in Agentic Systems

INFUSE is an end-to-end benchmark for indirect prompt injection (IPI) across long-horizon agent workflows (shell commands, code changes, dependency installs, remote services). It has 40 realistic tasks and a taxonomy of 56 attack scenarios in five threat categories, with attacks injected via interaction surfaces encountered mid-task. Across 19 LLMs (GPT, Claude, Gemini, Llama, Gemma, Qwen) and 41,824 adversarial trajectories, mean attack success was 19.6% (LLM-judged) and 15.4% (heuristic), per-model 3.2–41.9%; guard models gave limited protection, while prompt-level hardening cut mean ASR from 58.2% to 14.3%.

LLMClaudeGemini+3

indirect prompt injectionINFUSE

AI Security· Added 17 Oct 2026, published 14 Oct 2026

en.cryptonomist.charticle🧠 AI

Google’s Gemini 4 Argon AI solves 12 simulated cyberattacks, up from five

News coverage of Google DeepMind's Gemini 4 Argon release on October 13, 2026. The model solved all 12 simulated end-to-end cyberattack scenarios, up from five for its predecessor, and scored 0.7% attack success on Gray Swan's Indirect Prompt Injection benchmark. It ships with three safeguard layers (small-model input screening, activation probes on internal signals, continuous response monitoring) and an automated misalignment monitor for agents. Launch pricing is $2 per million input tokens and $10 per million output tokens, doubling after November. Access is through Pro and Ultra tiers, and Ultra also gets Antigravity. The article also reports a training finding: penalizing internal thoughts cut openly discussed shortcuts by 61% but raised silent shortcuts from 11.2% to 36.4%.

GeminiGoogle Antigravity

Gemini 4 ArgonGoogle DeepMindGoogle+7

models, ai-safety, evals· Published and added 14 Oct 2026

embracethered.comwriteup🛡️ Security

Visual Studio Copilot and SSMS Remote Code Execution (CVE-2026-103572)

CVE-2026-103572 is a remote code execution bug in Microsoft.VisualStudio.Copilot.Core.dll, shared by Visual Studio and SQL Server Management Studio. A trailing dot (for example `.mcp.json.`) bypasses the protected-filename check in EditsFunctionHandler.GetEditConfirmationTypeAsync, so a hijacked Copilot can rewrite its own MCP and settings files and run commands. The post covers path traversal and InstallationID leaks, indirect prompt injection demos, the October 2026 patch (GetFullPath), and a $0 bounty.

Visual StudioSQL Server Management StudioGitHub Copilot+3

CVE-2026-103572indirect prompt injectionpath traversal+5

AI Security, AppSec, Offensive Security· Added 29 Oct 2026, published 13 Oct 2026

deepmind.googlearticle🧠 AI

Gemini 4 Argon

Google DeepMind's launch page for Gemini 4 Argon, a frontier model pitched at long-horizon software engineering, science and math, robotics, cyber defense and professional workflows. Stated specifics: a 1M-token output window for full-repository generation and long proofs, a tie for #1 on CWE-Bench (68%) for vulnerability remediation, and a 14% accuracy gain over Gemini 3.1 Pro in Box's evaluation. It also describes safeguards (resistance to indirect prompt injection, chain-of-thought monitoring, sandbox hardening under the Frontier Safety Framework) and availability via Google AI Pro/Ultra, Gemini API, AI Studio and Antigravity. Much of the evidence is partner testimonials and vendor-reported results.

GeminiGoogle AntigravityGoogle AI Studio+1

Gemini 4 ArgonGoogle DeepMindGoogle+16

models, coding-agents, multimodal· Published and added 13 Oct 2026

arxiv.orgpaper🛡️ Security

Towards Safe Autonomy for Computer Use

Introduces SA-CUA, a dual-LLM framework that gives Computer Use Agents a trust boundary against indirect prompt injection. It derives conservative deterministic rules from the accessibility tree to unmask trusted screen regions for the privileged LLM, and adds an endorsement channel that moves values from untrusted to trusted domains by verification. Human input is needed only at task specification. It reports much higher utility than CaMeL-CUA with a zero attack success rate.

LLM

SA-CUACaMeL-CUACaMeL+2

AI Security· Added 15 Oct 2026, published 13 Oct 2026

arxiv.orgpaper🧠 AI

Towards Safe Autonomy for Computer Use

Introduces SA-CUA, a dual-LLM framework for Computer Use Agents (CUAs) that defends against indirect prompt injection. It builds a trust boundary by deriving conservative deterministic rules from the accessibility tree, which unmask trusted screen regions in the privileged LLM's screenshot and keep untrusted content separate. An endorsement channel lets values move from the untrusted to the trusted domain through verification over trusted channels. This removes the need for per-action user confirmation and the confirmation fatigue it causes. SA-CUA is reported to keep much higher utility than CaMeL-CUA with a 0% attack success rate.

LLM

SA-CUACaMeL-CUACaMeL

agents, research· Added 15 Oct 2026, published 13 Oct 2026

x.comthread🛡️ Security

Nicholas Michael Kloster (@showxlate) on X

A researcher announces a zero-click indirect prompt injection RCE in REA (Reverse Engineering Assistant), an MCP tool. Analyzing a malicious web target silently exfiltrates AWS keys, GitHub tokens and SSH identities. The stated root cause is that process.env is passed unconditionally to child processes, so the full environment is exposed to attacker-driven commands. A full writeup is promised but not yet published; replies add unverified skepticism about the project's provenance.

Model Context ProtocolNode.jsAWS+2

indirect prompt injectionREANicholas Michael Kloster+1

AI Security, AppSec· Added 13 Oct 2026, published 12 Oct 2026

arxiv.orgpaper🛡️ Security

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

This arXiv paper introduces RAISED (Robust Attack Invariance through Self-Distillation), a training framework that defends tool-using LLM agents against indirect prompt injection without the utility degradation seen in prior training-based defenses. The authors show existing defenses cause distributional drift that makes models wrongly refuse legitimate tool-output-guided steps on benign tasks, then fix this by having a model generate its own tool-use scenarios and self-distill to match clean-context behavior on both clean and injected trajectory variants. Results show reduced attack success rate on injected tool responses while preserving performance on agentic and general-purpose benchmarks.

LLM

RAISED

AI Security· Added 6 Oct 2026, published 5 Oct 2026

arxiv.orgpaper🛡️ Security

Who Is Your Agent Serving? Provider-Side Indirect Prompt Injection in Proactive Agents

This arXiv paper identifies a new attack surface in proactive personal AI agents: an external content provider who controls only material tied to its own target (e.g. a product or service) can manipulate an otherwise benign agent into recommending that target to the user, without compromising the agent or accessing private user context. The authors formalize the attack via three mechanisms — Target Control, Private Binding, and Prospective Support — and show across three proactive-agent environments and six simulated user models that the full attack raises target authorization rates by up to 77.4 percentage points, with a multi-turn variant that keeps influencing the agent's responses even absent final user authorization.

Target ControlPrivate BindingProspective Support

AI Security· Added 6 Oct 2026, published 4 Oct 2026

decrypt.coarticle🧠 AI

Gemini 4 Is Here, and Google’s Flagship Tops All Other AI Models on Cybersecurity

Google released Gemini 4 Argon, its new frontier model for coding, office work and cyber defense, scoring 77.9% on DeepSWE v1.1 (vs. 74.2% Claude Opus 5.5, 74.1% GPT-6 Astra) and expanding max output to 1 million tokens. On Gray Swan's indirect prompt injection benchmark it posted a 0.7% attack success rate, edging out Claude Opus 5.5 and Claude Fable 5.1 (both 1.0%), and scored 70.9% on Google's internal Wiz Penetration Test Benchmark versus 58.2% for the earlier restricted Gemini 3.8 Flash Cyber. The model initially ships without cyber guardrails only to vetted defenders via Google's Fairwind Program before wider API and Google AI Ultra release at $2/$10 per million input/output tokens.

Gemini

GoogleClaude Opus 5.5Claude Fable 5.1+16

models, evals· Published and added 30 Sep 2026

decrypt.coarticle🛡️ Security

Gemini 4 Is Here, and Google’s Flagship Tops All Other AI Models on Cybersecurity

Decrypt covers Google's Gemini 4 Argon launch, reporting it scored a 0.7% attack success rate on Gray Swan's Indirect Prompt Injection benchmark, beating Claude Opus 5.5 and Claude Fable 5.1 (both 1.0%), GPT-6 Astra (8.5%), and far outperforming Grok 4.6 and Kimi K3 (~52% each). It also details Argon's cyber-defense capability: it ships first to vetted defenders through Google's Fairwind Program "without cyber guardrails," scored 70.9% on the internal Wiz Penetration Test Benchmark writing working exploits against real web-app flaws, and reportedly helped Wiz find a critical healthcare-software flaw earlier models missed.

GeminiClaudeOpenAI+3

Fairwind ProgramGray SwanGemini 4 Argon+8

AI Security· Published and added 30 Sep 2026

embracethered.comwriteup🛡️ Security

From SELECT to SYSADMIN with SQL Copilot (CVE-2026-65669)

Write-up of a BlueHat Asia 2026 talk on CVE-2026-65669, a critical SQL Server elevation of privilege in Copilot in SQL Server Management Studio. The read-only mode was only a regex classifier (LocalSqlExecutionAccessChecker) and was bypassed with DECLARE @p sysname='sp_executesql'; EXEC @p. The chain used indirect prompt injection through AGENTS.md and CONSTITUTION.md extended properties to take a db_owner to sysadmin, with data exfiltration via xp_dirtree and RestoreVerifyBackupFile.

SQL Server Management StudioMicrosoft SQL ServerCopilot in SSMS+2

CVE-2026-65669indirect prompt injectionprompt injection+9

AI Security, AppSec, Offensive Security· Added 29 Oct 2026, published 30 Sep 2026

arxiv.orgpaper🛡️ Security

Prompt Injection Detection for Email Agents Through Attack Chain Modeling

This paper proposes an attack-chain-based detection framework for indirect prompt injection in LLM email assistants, combining a text detector, stage-specific verifiers, rule-based risk signals, and user-intent/action consistency checks scored via a logistic decision policy, rather than treating detection as binary malicious-text classification. Evaluated under random splits, temporal and cross-dataset transfer, and ablations across five binary benchmarks, it achieves a mean F1 of 0.406 versus 0.216 for the best of five existing pretrained detectors, and shows random splits overestimate robustness under distribution shift while training on hard benign (attack-resembling) emails reduces false alarms.

LLM

AI Security· Added 30 Sep 2026

arxiv.orgpaper🛡️ Security

From A2A Attacks to Envelope-Layer Defense: Red-Teaming Evaluation of LLM Agents and a Three-Layer Isomorphic Attack-Defense Model

This paper proposes A2A-TIBA, an indirect prompt injection attack that abuses agent-to-agent (A2A/ACP) protocols by getting a target LLM agent to deploy a callback program, letting an attacker bypass the agent entirely to issue commands and exfiltrate data. It also introduces GDA Measurement, a red-team testbed that captures raw gateway context to distinguish four outcome classes (semantic refusal, semantic breach, interception, penetration) instead of a single attack-success-rate metric, and ELA-ITL, a three-layer isomorphic attack-defense model splitting defense into envelope packaging, LLM recognition, and agent interception. Testing on 15 agent-frontend x LLM-backend combinations and a 1,000-case dataset shows that labeling malicious-prompt indicators in envelope channels (A2A, tool, memory) significantly improves LLM recognition of injected content.

LLM

A2A-TIBAGDA MeasurementELA-ITL

AI Security· Added 2 Oct 2026, published 30 Sep 2026

arxiv.orgpaper🛡️ Security

Tracekit: Tamper-Evident Intent-Reasoning-Action Auditing for Autonomous Coding Agents

Tracekit is an open-source, dependency-free system that logs three cross-checked channels (human intent, model self-reported reasoning, and actual executed actions) for autonomous coding agent sessions into a hash-chained, externally anchorable ledger, hooking into Claude Code's lifecycle events and reconstructing multi-agent hierarchies. The paper's experiments show the ledger detects all 1,600 tested tamper mutations (edits, deletions, reordering, forged inserts, torn writes) with anchoring needed to catch tail truncation/full re-chaining, that a regex-based tool-call gate blocks only 41% of harmful calls while over-blocking benign ones and is trivially evaded, that in 14 real Claude Code runs the agent never acted on 4 planted indirect prompt injections, and that an LLM reviewer catches 100% of seeded misaligned steps in traces versus 82% for rule-based flags. The authors release the system, harness, and all traces.

Claude Code

AI Security, Blue Team & Detection· Added 29 Sep 2026, published 28 Sep 2026

arxiv.orgpaper🛡️ Security

Certified Multi-Source Integrity for Structured Agent Actions

This arXiv paper proposes a certifier for structured, privileged agent actions such as paying an invoice, assembled from potentially corrupted documents and tool outputs, formalizing when multi-source corroboration can be trusted under indirect prompt injection. It uses a corruption-radius rule counted via minimum hitting sets over corruption-distinct evidence classes so laundered or re-published copies cannot fake a quorum, plus deterministic reconciliation and trusted anchors for single-sourced fields. Evaluated on 70,966 sanctions entities and 450 software-supply-chain provenance records, and against five LLM agents where a realistic injection fools all but one model and existing action-gating or provenance baselines are broken while the certifier admits no unsafe action.

LLM

AI Security· Added 2 Oct 2026, published 28 Sep 2026

github.comrepo🛡️ Security

rewanthtammana/Who-Let-The-Agents-Act

who-let-the-agents-act is a self-hostable lab suite (Docker/Python, Groq-backed LLM agent) with nine isolated scenarios — overpowered data tools, IDOR-style cross-account access, unsafe agent handoffs, refund-limit bypass, indirect prompt injection via poisoned invoices, multi-tenant RAG leakage, secret leakage through debugging, fail-open approval outages, and a multi-hop confused-deputy agent chain. Each scenario runs in multiple modes (vulnerable vs. application-enforced) against the same prompt, recording model plan, policy decision, tool calls, and DB access as JSON so a reader can directly compare what changes when authorization is enforced outside the system prompt.

PythonDocker

IDOR

AI Security, AppSec· Added 26 Sep 2026

arxiv.orgpaper🛡️ Security

No-Box Vulnerability Analysis: Description-only Detection of Indirect Prompt Injection Vulnerabilities in MCP Servers

This paper proposes no-box vulnerability analysis, a new paradigm for auditing systems using only functionality metadata when neither system access nor runtime interaction is available. The authors build MCPSEC, a prototype that audits Model Context Protocol servers for indirect prompt injection vulnerabilities purely from tool metadata exposed at server registration, evaluating it against 20 widely-deployed MCP servers with 177 tools where human evaluators confirmed 95 vulnerable tools; MCPSEC achieved 98.9% recall versus 84.2% for an LLM baseline, also producing hypothesized exploitation techniques per finding.

Model Context ProtocolLLM

MCPSEC

AI Security· Added 26 Sep 2026

arxiv.orgpaper🛡️ Security

ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack

ReasAlign is a model-level safety alignment technique that defends LLM agents against indirect prompt injection by adding structured reasoning steps to detect conflicting instructions embedded in external data and preserve the user's original task intent. It also uses a test-time scaling mechanism with a preference-optimized judge model to score and select the best reasoning trajectory. On the CyberSecEval2 benchmark it reports 94.6% utility with only 3.6% attack success rate, substantially outperforming Meta SecAlign, which scores 56.4% utility with a 74.4% attack success rate.

LLM

Meta SecAlignCyberSecEval2

AI Security· Added 22 Sep 2026

wiz.ioarticle🛡️ Security

Building an AI Detection Engine That Understands Agent Intent

Wiz describes an AI detection engine that analyzes model input/output telemetry (prompts, system prompts, tool calls, reasoning) rather than just outputs, in order to catch shifts in AI agent intent such as indirect prompt injection. It walks through OpenAI's disclosed Hugging Face/ExploitGym incident and an internal support-agent phishing-relay scenario, then describes a staged pipeline (regex/SLM candidation, light LLM filtering, deep LLM analysis, correlation with cloud/runtime telemetry) validated against a benchmark of 100+ simulated scenarios covering 57 injected attacks and 45 benign workflows across CI/CD, DevOps, data analyst, support, CLI, and cloud auditor agent personas.

WizOpenAIHugging Face+1

ExploitGym

AI Security, Blue Team & Detection· Published and added 17 Sep 2026

github.comrepo🛡️ Security

AI Agent Hacking Writeups

A GitHub-hosted, CC0-licensed index of 281 public writeups on attacks against AI agents, organized into six categories: indirect prompt injection, direct prompt injection/jailbreaks/guardrail bypass, MCP and agent tooling, AI platform/infrastructure bugs, classic web bugs in AI product surfaces, and AI used as an attacker's tool. Entries span 2021-2026 and include major disclosures like GeminiJack, ForcedLeak, Clinejection, CVE-2025-53773 (GitHub Copilot RCE), and dozens of Johann Rehberger/PromptArmor/qual1a writeups covering exfiltration, MCP OAuth flaws, and coding-agent sandbox escapes. Comes with an accompanying blog post "How to Hack AI Agents" and a scripted links.txt for automation.

Claude CodeModel Context ProtocolGitHub Copilot+17

CVE-2026-24299CVE-2025-66032CVE-2025-62453+25

AI Security· Added 22 Sep 2026, published 17 Sep 2026

endorlabs.comarticle🛡️ Security

Cursor Security: How to Secure AI-Generated Code in 2026 | Blog | Endor Labs

Endor Labs blog post detailing seven security risks specific to Cursor's AI code generation that its built-in controls (workspace trust, SOC 2, privacy mode) don't cover: indirect prompt injection via files/comments, malicious .cursorrules files creating persistent backdoors, AI-suggested typosquatted or outdated dependencies, hardcoded credential leakage in generated test/boilerplate code, cross-project context poisoning via repository indexing, and unreviewed agent auto-run execution. It contrasts Cursor's native controls with what external AppSec tooling (pitched as Endor Labs' AURI) needs to fill, recommending dependency pinning, egress filtering, secret-scanning pre-commit hooks, and reachability-based SCA/SAST.

CursorEndor Labs

AURI

AI Security, Supply Chain· Published and added 11 Sep 2026

endorlabs.comarticle🛡️ Security

Prompt Injection Against Coding Agents: The Attack Surface Nobody Owns | Blog | Endor Labs

Endor Labs breaks down how indirect prompt injection compromises AI coding agents (Cursor, GitHub Copilot) via poisoned READMEs/rules files, malicious MCP servers, and web/issue content the agent reads with elevated trust and auto-approve permissions. Cites a study testing 314 payloads across 70 MITRE ATT&CK techniques showing up to 84% attack success, OWASP's MCP tool poisoning definition, Unit 42's real-world IDPI telemetry, and CVE-2025-52573 as a concrete command-injection-via-prompt-injection case in an MCP server.

Model Context ProtocolCursorGitHub Copilot

CVE-2025-52573MITRE ATT&CKUnit 42+2

AI Security· Added 3 Sep 2026, published 2 Sep 2026

embracethered.comwriteup🛡️ Security

Breaking Claude Code Opus 5 Auto Mode · Embrace The Red

Embrace The Red (Johann Rehberger) demonstrates an indirect prompt injection attack chain that achieves 60-80% success rates hijacking Claude Code Opus 5 Auto Mode into remote code execution, contrasting with a vendor-run benchmark that reported 0.00% attack success. The chain nudges Claude from WebFetch to curl, delivers a malicious ZIP archive, and exploits Python module shadowing (a poisoned struct.py) when Claude writes its own decoder script, ultimately downloading and executing a payload that establishes C2 and even spawns a nested headless Claude Code instance. Anthropic closed the report as "Informative, working as designed," framing Auto Mode as a best-effort classifier rather than a security boundary, reinforcing that sandboxing and egress control remain necessary mitigations.

Claude CodePythoncurl+2

Johann RehbergerBoris ChernyTrajectory Labs+4

AI Security, Offensive Security· Added 31 Aug 2026

varonis.comwriteup🛡️ Security

CoSnitch: When Your AI Assistant Becomes Its Own Whistleblower (CVE-2026-24301)

Varonis Threat Labs details CoSnitch (CVE-2026-24301), a chain of three flaws in Microsoft Copilot Personal. A crafted copilot.microsoft.com link with ?q= and an undocumented autorun parameter runs a prompt on load. That prompt reads data through OAuth connectors (Gmail, Drive, Calendar) and exfiltrates it via Copilot's URL-fetch to an attacker webhook. Indirect prompt injection through web summarization also writes persistent attacker instructions into Copilot's memory. The post also describes "meta-hacking", where researchers got Copilot to disclose its own hidden parameter, and lists mitigations.

Microsoft CopilotOAuthGmail

CVE-2026-24301CoSnitchmeta-hacking+3

AI Security, AppSec· Added 5 Sep 2026, published 18 Aug 2026

adversa.aiarticle🛡️ Security

Top 10 zero-click attacks against AI agents

A blog post from AI-security firm Adversa AI cataloging ten zero-click attack vectors against AI agents — attacks that compromise an agent without requiring any user interaction, likely covering vectors such as malicious tool outputs, poisoned context/documents, or indirect prompt injection delivered via content the agent autonomously ingests. Adversa AI regularly publishes original research on LLM jailbreaks and agent security, making this a candidate reference for agentic-AI attack taxonomies.

AI Security· Added 7 Oct 2026, published 18 Aug 2026

securityweek.comarticle🛡️ Security

Zero-Click AI Browser Hacking: Claude and ChatGPT Atlas Hijacked via Emails, X Posts

SecurityWeek covers Zenity's 'PleaseFix' research disclosing zero-click indirect prompt injection attacks against OpenAI's ChatGPT Atlas and Anthropic's Claude in Chrome browser extension. In Atlas, a single planted X comment triggers 'intent collision' that resurrects CSRF-like cross-site abuse, letting the agent read WhatsApp Web contacts to send phishing messages or manipulate Amazon carts and use Amazon's Rufus assistant to complete unauthorized purchases. In Claude in Chrome, a malicious email with hidden prompt instructions triggers the agent to import a payload from a rogue NPM package hosted on a fake CDN, enabling exfiltration of Gmail inbox contents via the Atom feed, mass-sharing of Google Drive files, and full account takeover of Slack and X by intercepting MFA codes.

ChatGPTClaudeChrome+3

ZenityPleaseFixintent collision+3

AI Security· Added 26 Sep 2026, published 6 Aug 2026

adversa.aiarticle🛡️ Security

Top MCP security resources — August 2026

Adversa AI's monthly MCP security digest for August 2026, grouping ten resources: the July 28 MCP specification revision that hardens authorization (RFC 9207 issuer validation, deprecation of Dynamic Client Registration, a stateless core with header-based routing), a confused-deputy indirect prompt injection against Microsoft's Azure DevOps MCP server that hijacks a review agent into exfiltrating wiki data, SANS ISC telemetry showing 49 source IPs scanning for exposed MCP servers and agent credentials in two weeks, and new research (ChainWatch's kill-chain detection via Hidden Markov Models, SPELLSMITH's description-layer mitigations for taint-style flaws, MTGuard's static/dynamic tool-call analysis, and a 33-vulnerability taxonomy of agentic commerce platforms with the AIP-Bench benchmark). It closes with concrete upgrade guidance: audit DCR dependencies, validate issuers per RFC 9207, grep logs for MCP scanning indicators, and stand up a policy-enforcement gateway.

Model Context ProtocolOAuthAzure DevOps

RFC 9207SANS ISCChainWatch+5

AI Security, Identity & Access, AppSec· Added 7 Oct 2026, published 6 Aug 2026

aws.amazon.comadvisory🛡️ Security

CVE-2026-18394 - Incorrect authorization in Strands Agents Tools http_request tool

AWS bulletin for CVE-2026-18394, an incorrect authorization flaw in the http_request tool of the open-source strands-agents-tools package used with the Strands Agents SDK. The tool exposed an LLM-controllable proxies parameter that let attacker-controlled web content (via indirect prompt injection) redirect requests through an actor's proxy while the hostname allowlist check still passed, disclosing HTTP_REQUEST_TOKEN_CONFIG-bound credentials in cleartext Authorization headers on the first hop. Fixed in version 0.8.2; AWS recommends upgrading and rotating any credentials configured on affected versions.

AWSStrands AgentsLLM

CVE-2026-18394

AI Security, AppSec· Added 6 Aug 2026

forcepoint.comwriteup🛡️ Security

Beyond Prompt Injection: Persistent Memory Poisoning in AI Agents

Forcepoint X-Labs details persistent memory poisoning, an attack where indirect prompt injection (e.g. hidden text in a webpage) tricks an AI agent into storing attacker-controlled facts, like a fake ABC Travel Support booking provider, in its long-term memory, so the poisoned content is later retrieved and trusted across unrelated future sessions. The article proposes a memory risk-scoring defense (source reputation, remember-this phrasing detection, new-contact or domain checks, contradiction detection, sensitive-category flags) and ships a working Python proof-of-concept comparing a vulnerable agent that blindly trusts extracted memory against a defended agent that scores and quarantines risky memory objects before use.

Python

AI Security· Added 26 Sep 2026, published 4 Aug 2026

securityweek.comarticle🛡️ Security

AI Security Briefing

SecurityWeek reports on Zscaler research uncovering two indirect prompt injection campaigns aimed at AI agents: one uses SEO-poisoned fake documentation for a Python package (requests-secure-v2) to trick agents into making cryptocurrency payments for a fake "API key," hiding instructions in schema markup and hidden HTML divs; the other typosquats the DeFi platform DeBank to get agents to trust a fraudulent site as legitimate. Zscaler built an autonomous payment-capable AI agent and tested 26 LLMs, finding four (Llama 3.3 70B Instruct, Llama 3.2 90B Vision Instruct, Gemini 3 Flash, Gemini 2.5 Pro) could be manipulated into paying, while Claude Sonnet 4.5 and GPT-5.4 misidentified the fake DeBank site as legitimate.

PythonPyPIEthereum+4

DeBank

AI Security, Threat Intel· Published and added 18 Jul 2026

cybersecuritynews.comadvisory🛡️ Security

AI Security Briefing

CVE-2026-14898 affects the OpenAI Codex desktop app for macOS, which automatically renders remote images embedded in Markdown model output without user interaction. An attacker can use indirect prompt injection to make Codex generate a response containing an attacker-controlled image URL whose parameters embed sensitive data (API keys, source code, tool output), which is silently exfiltrated when the app auto-fetches the image. Categorized under CWE-200, no patch or CVSS score was available at disclosure.

OpenAI CodexmacOS

CVE-2026-14898CWE-200indirect prompt injection

AI Security, AppSec· Published and added 18 Jul 2026

zscaler.comwriteup🛡️ Security

AI Security Briefing

Zscaler ThreatLabz documents two indirect prompt injection (IPI) campaigns targeting AI agents: a fake Python package documentation site (spoofing "requests-secure-v2") that hides instructions in CSS-concealed divs and JSON-LD structured data to trick agents into paying a fake $3 API license fee via Stripe or Ethereum, and a typosquatted DeBank clone (debank[.]auction) that uses hidden prompts to convince agents it is the authoritative DeFi platform. Testing an autonomous web-enabled agent across 26 LLMs, 4 models (Llama 3.3 70B, Llama 3.2 90B Vision, Gemini 3 Flash, Gemini 2.5 Pro) executed the fraudulent payment, and 2 models (GPT-5.4, Claude Sonnet 4.5) misclassified the typosquatted site as legitimate depending on context provided.

LlamaGeminiClaude+2

ThreatLabzDeBankOpen-Agent-Utilities+1

AI Security, Threat Intel· Published and added 18 Jul 2026

github.comrepo🛡️ Security

llm-security (greshake)

GitHub repo by Kai Greshake collecting research and proof-of-concept material on LLM security, notably indirect prompt injection attacks where malicious instructions are embedded in retrieved content (web pages, documents) to hijack LLM-integrated applications like Bing Chat. It underpins some of the earliest published demonstrations of prompt injection as a practical attack vector against production AI assistants.

ChatGPTBing Chat

Kai Greshake

AI Security· Added 3 Sep 2026, published 14 Jul 2026

github.comrepo🛡️ Security

Awesome LLM Security

Awesome LLM Security is a GitHub-curated list of papers, tools, and datasets covering LLM attack and defense research, including jailbreak techniques (GCG, AutoDAN, DeepInception, GPTFUZZER, cipher-based jailbreaks), indirect prompt injection, backdoor/virtual prompt injection attacks, model fingerprinting (LLMmap, TRAP), and defenses like PARDEN, circuit breakers, and self-defense filtering. It's a comprehensive reference for tracking the state of adversarial and defensive LLM security research from 2022 onward.

ChatGPTGPT-4

Visual Adversarial ExamplesGCG (Universal and Transferable Adversarial Attacks)AutoDAN+8

AI Security· Added 3 Sep 2026, published 14 Jul 2026

embracethered.comwriteup🛡️ Security

Copirate 365 at DEF CON: Plundering in the Depths of Microsoft Copilot (CVE-2026-24299) · Embrace The Red

Johann Rehberger's writeup of his DEF CON Singapore talk detailing a multi-stage exploit chain against Microsoft 365 Copilot and Consumer Copilot (CVE-2026-24299): indirect prompt injection triggers Copilot's HTML preview feature, CSS background-image/font-face rules bypass img-src CSP and exfiltrate data over font-src, and the attack is made zero-click by instructing Copilot to auto-open the preview. It further chains this with abuse of Copilot's record_memory / durable-facts long-term memory tools (which have no audit logging) to plant persistent "SpAIware"-style instructions that silently exfiltrate any password or secret the user later types, plus Delayed Tool Invocation as a reliability trick and edge_navigate_to abuse in Edge's Copilot to leak stored memories. Includes full disclosure timeline (Oct 2025 report to Dec 2025/Mar 2026 fixes) and broader lessons on CSP not being a reliable AI security boundary and the "lethal trifecta."

Microsoft 365Microsoft CopilotMicrosoft Edge

CVE-2026-24299Johann Rehbergerlethal trifecta+4

AI Security, Offensive Security· Added 6 Jul 2026

youtu.bevideo🛡️ Security

Web LLM Attacks Lab: Exploiting AI agents to trigger secondary vulnerabilities

This walk-through demonstrates solving PortSwigger Web Security Academy's 'Exploiting AI agents to trigger secondary vulnerabilities' lab. The attack leverages Burp Suite to discover an internal administrative endpoint via SSRF fuzzing, then plants an indirect prompt injection payload in a public product review to coerce an automated AI site auditor into deleting a target user.

PortSwigger Web Security Academy Lab - Exploiting AI agents to trigger secondary vulnerabilitiesBurp Suite Professional v2024.2.1Burp Intruder+4

PortSwigger Web Security Academy

AI Security· Added 3 Oct 2026, published 2 Jun 2026

youtu.bevideo🛡️ Security

Web LLM Attacks Lab: Bypassing AI Scanner Defenses to Exfiltrate Sensitive Information

This walkthrough demonstrates how an indirect prompt injection vulnerability in an automated AI-powered security scanner can be exploited to exfiltrate sensitive API credentials via a public blog comment section. By posting instructions masquerading as a vulnerability advisory, an attacker tricks the AI scanner into logging in and posting its valid API key publicly.

PortSwigger Web Security AcademyOBS StudioLLM

AI Security· Added 3 Oct 2026, published 2 Jun 2026

youtu.bevideo🛡️ Security

Web LLM Attacks Lab: Exploiting AI agents to exfiltrate sensitive information

Demonstrates solving the PortSwigger Web Security Academy lab on bypassing AI scanner defenses via indirect prompt injection. An attacker injects a malicious prompt into blog comments that tricks an automated AI security auditor into retrieving and publishing its API key in plaintext.

PortSwigger Web Security Academy - Bypassing AI scanner defenses to exfiltrate sensitive informationPortSwigger Web Security AcademyOBS Studio+1

PortSwigger

AI Security· Added 3 Oct 2026, published 2 Jun 2026

cyberscoop.comadvisory🛡️ Security

Careful Adoption of Agentic AI Services

Joint guidance co-authored by ASD's ACSC, CISA, NSA, the Canadian Centre for Cyber Security, NCSC-NZ and NCSC-UK on securing LLM-based agentic AI systems across critical infrastructure and defence. It catalogs agentic-AI-specific risk categories — privilege compromise, scope creep, confused-deputy attacks, identity spoofing, insecure static permission checks, indirect prompt injection, specification gaming and deceptive/emergent agent behaviour — with concrete scenario examples, and lays out best practices for designing, developing, deploying and operating agents securely.

LLM

ASD's ACSCCISANSA+6

AI Security· Added 3 Sep 2026, published 13 May 2026

github.comrepo🛡️ Security

anmolksachan/AI-ML-Free-Resources-for-Security-and-Prompt-Injection

A GitHub-hosted, structured learning roadmap for AI/ML security and prompt injection, covering prerequisites (web/API basics), LLM fundamentals (OWASP LLM Top 10 2025, MITRE ATLAS), attack classes (prompt injection, indirect prompt injection, jailbreaking, multi-turn attacks, MCP tool poisoning, RAG poisoning, model extraction, supply chain), and hands-on labs/tools (Garak, PyRIT, ART, CleverHans, ModelScan). It also catalogs named CVEs in AI coding assistants (CVE-2025-53773 GitHub Copilot RCE, CVE-2025-54135 Cursor IPI-to-RCE) and points to bug bounty platforms, checklists for pentesting LLM apps, and community resources (AI Village, Embrace the Red, r/llmsecurity).

Model Context ProtocolGitHub CopilotCursor+3

CVE-2025-53773CVE-2025-54135DAN (Do Anything Now)+3

AI Security, Learning· Published and added 7 May 2026