36 results for “Attack Success Rate”

List Grid

What you seeeverything

Recent Oldest Added A–Z

All types36 Papers29 Repos3 Articles2 Write-ups2

“Attack Success Rate”

arxiv.orgpaper🛡️ Security

REALM: Identifying Minimal Task Assets for LLM Agents

REALM (Required Execution Assets Linked to a Mission) is an approach to applying least privilege to LLM agents by identifying the assets a task needs, including ones discovered at runtime. It models assets as Dependency Graph nodes, separates discovering an asset from authorizing it, and exposes resource-bound Partial APIs inside a Task Container. On AgentDyn with qwen3.5-flash, attack success rate falls from 16.6% to 0.2%, with 40.0% task completion on clean tasks and 36.4% on injected cases.

QwenLLMAgentDyn

prompt injectionleast privilegeREALM

AI Security· Added 31 Oct 2026, published 29 Oct 2026

arxiv.orgpaper🛡️ Security

Post-Training Large Language Models with LLMON

LLMON (LLM Object Notation) is a framework that separates instructions from data and requires explicit identifier-based execution. The authors post-train Granite-4.1 3B and 8B base models with SFT then RL on a 4.6B-token structured corpus covering general, RAG and tool-calling data, with distractor variants. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, and general capability is preserved. Attention-based metrics diagnose failures per transformer layer.

LLMONGranite 4.1LLM

prompt injectionindirect prompt injection

AI Security· Added 30 Oct 2026, published 28 Oct 2026

arxiv.orgpaper🧠 AI

Post-Training Large Language Models with LLMON

LLMON (LLM Object Notation) is a framework that separates executable instructions from passive data and requires explicit identifier-based execution. The paper post-trains Granite-4.1 3B and 8B base models on a 4.6B-token corpus of general, RAG and tool-calling data with distractor variants, using SFT then RL. Attack success rates on indirect prompt-injection benchmarks fall from 53-100% to 0.02-5.6%, with general capabilities preserved. It also adds attention-based per-layer diagnostics.

LLMRAGTransformer

LLMONGranite-4.1arXiv

training, research· Added 30 Oct 2026, published 28 Oct 2026

eprint.iacr.orgpaper🛡️ Security

For Thirty Pieces of Silver: Overcoming Non-IID via ZKP Sample-Level Auditing and Forcing Attackers into Traitors in Federated Learning

Preprint (IACR ePrint 2026/2611) proposing a federated learning defense that replaces gradient detection with zero-knowledge-proof sample-level audits of client datasets. A staking mechanism forces attackers to train on discovered poisoned samples with flipped labels, producing counter-gradients that remove the backdoor without knowing the trigger. ZKP commitments lock the training data, which blocks adaptive attacks and gradient tampering. On CIFAR-10 under Dir(0.1) non-IID, it holds attack success rate below 6% against distributed backdoor and edge-case attacks.

distributed backdoor attackedge-case attackadaptive attack+3

AI Security· Added 24 Oct 2026, published 23 Oct 2026

arxiv.orgpaper🛡️ Security

Concerns in Benchmarking Jailbreaks: An Empirical Study of Attack Success Rates

This arXiv paper tests eight jailbreak attacks against three open-weight language models under three evaluation procedures. Low-resource language attacks score 96.0% mean ASR under keyword matching but 2.7% under a Qwen3 judge using StrongREJECT instructions. At Ollama's default temperature of 0.8, 100 samples of an unchanged request give 100% ASR by keywords and 50% by the judge. The authors recommend explicit success definitions, matched response budgets, validated evaluators and cumulative-failure reporting.

OllamaQwenLLM

StrongREJECTDeepInceptionattack success rate+1

AI Security· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🧠 AI

Concerns in Benchmarking Jailbreaks: An Empirical Study of Attack Success Rates

Concerns in Benchmarking Jailbreaks is an October 2026 arXiv study of eight jailbreak attacks on three open-weight models under three evaluation procedures. Low-resource language attacks score 96.0% ASR by keyword matching but 2.7% under a Qwen3 evaluator using StrongREJECT instructions. Attack rankings flip with attempt budgets, and 100 samples at Ollama's default temperature 0.8 give 100% ASR by keywords, 50% by the evaluator.

OllamaQwen3LLM

StrongREJECTDeepInceptionarXiv

evals, ai-safety, research· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🛡️ Security

SpanGuard: Decoupling Instruction Localization from Authorization for Lightweight IPI Defense

SpanGuard is an indirect prompt injection (IPI) guardrail that splits detection into two steps. Lightweight unsupervised segmentation first locates candidate instruction blocks in tool outputs, then a small LLM judges whether each block is authorized by the user's request. The authors report a 0.32% attack success rate with negligible benign utility loss, and the guardrail still works with substantially smaller models.

LLM

indirect prompt injectionprompt injection

AI Security· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🛡️ Security

RAGLeak: Supply-Chain Privacy Backdoors in Retrieval-Augmented Generation

RAGLeak is a supply-chain privacy backdoor attack on retrieval-augmented generation. When triggered, the backdoored generator reversibly obfuscates retrieved private documents, and an auxiliary LLM recovers them. Tested on three model families and three domains, it reaches up to 100% attack success rate, loses at most 10 points under detection-based defenses, and survives clean fine-tuning and changes of model scale and version. Source code is released.

LLM

RAGLeak

AI Security, Supply Chain· Added 21 Oct 2026, published 19 Oct 2026

arxiv.orgpaper🧠 AI

Ignored, Not Refused: What Low Prompt-Injection Success Rates Hide in Small Tool-Using Agents

"Ignored, Not Refused" audits 33,000 trajectories from eleven open-weight agents (2-14B parameters) on InjecAgent and AgentDojo. A label-blind LLM judge finds 76-80% of failed injection attacks were ignored and 17-18% refused. Changing only the system prompt moves InjecAgent attack success rate from 0.2% to 43%, and the ATOM matrix shows 21.7% of AgentDojo episodes were safe but left the user's task undone. The corpus, judge prompt and code are released.

LLM

InjecAgentAgentDojoATOM+1

evals, ai-safety, agents· Added 20 Oct 2026, published 16 Oct 2026

arxiv.orgpaper🛡️ Security

Towards Safe Autonomy for Computer Use

Introduces SA-CUA, a dual-LLM framework that gives Computer Use Agents a trust boundary against indirect prompt injection. It derives conservative deterministic rules from the accessibility tree to unmask trusted screen regions for the privileged LLM, and adds an endorsement channel that moves values from untrusted to trusted domains by verification. Human input is needed only at task specification. It reports much higher utility than CaMeL-CUA with a zero attack success rate.

LLM

SA-CUACaMeL-CUACaMeL+2

AI Security· Added 15 Oct 2026, published 13 Oct 2026

arxiv.orgpaper🧠 AI

Towards Safe Autonomy for Computer Use

Introduces SA-CUA, a dual-LLM framework for Computer Use Agents (CUAs) that defends against indirect prompt injection. It builds a trust boundary by deriving conservative deterministic rules from the accessibility tree, which unmask trusted screen regions in the privileged LLM's screenshot and keep untrusted content separate. An endorsement channel lets values move from the untrusted to the trusted domain through verification over trusted channels. This removes the need for per-action user confirmation and the confirmation fatigue it causes. SA-CUA is reported to keep much higher utility than CaMeL-CUA with a 0% attack success rate.

LLM

SA-CUACaMeL-CUACaMeL

agents, research· Added 15 Oct 2026, published 13 Oct 2026

arxiv.orgpaper🛡️ Security

Only What You've Signed up For: Secure Workflows with Constrained Diffusion Agents

Introduces Policy-Constrained Diffusion Agents (CoDA), which enforces task-dependent security policies inside the denoising process of diffusion language models. At each step, token distributions are projected onto a policy-compliant set, so constraints hold throughout generation instead of relying on detecting injected instructions. Evaluated across several dLLM families under static and adaptive prompt-injection attacks, it reports substantially lower attack success rates with competitive task utility and practical inference overhead.

diffusion language models

Policy-Constrained Diffusion AgentsCoDAprompt injection+1

AI Security· Added 15 Oct 2026, published 10 Oct 2026

arxiv.orgpaper🧠 AI

ReSI: Recursive Safety Improvement toward Resistant and Resilient AI

ReSI is a recursive safety improvement framework that automates rounds of safety alignment. Each round applies diverse red-teaming methods to the current target model, develops training recipes to fix the vulnerabilities found, and promotes the update with the largest safety gain among those passing a Pareto gate on capability retention. On four dense and mixture-of-experts models it matches or exceeds frontier models on in-distribution and out-of-distribution safety benchmarks. It cuts mean X-Teaming attack success rate from 86.01% to 31.45%, against 56.69% for the best frontier model (GPT-5.6 Luna), while largely preserving general capabilities.

Mixture of Experts

ReSIR²AIX-Teaming+1

ai-safety, research, training· Added 9 Oct 2026, published 8 Oct 2026

arxiv.orgpaper🛡️ Security

Package Hallucination Attacks on Coding Agents through Prompt Injection in Rule Files

This arXiv paper introduces the "package hallucination attack," where an attacker injects malicious prompts into benign agentic-coding rule files (such as .cursorrules) to induce a coding agent to replace legitimate package dependencies with attacker-controlled ones. The authors propose PackHallu, an evolutionary optimization framework that iteratively rewrites injected prompts using trajectory-level feedback and LLM-guided mutations, and show it achieves high attack success rates with strong transferability across multiple LLMs, benchmarks, and agent frameworks.

CursorLLM

PackHallu

AI Security, Supply Chain· Added 8 Oct 2026, published 7 Oct 2026

arxiv.orgpaper🛡️ Security

Secure Speculative Decoding for Large Language Models

This arXiv paper presents the first systematic study of security implications of speculative decoding, a technique that speeds up LLM inference using a smaller draft model to propose tokens verified by the target model. The authors find a security-utility asymmetry: lossy speculative decoding methods raise jailbreak and prompt-injection attack success rates much faster than they degrade utility, with security degradation traced to early tokens generated by the draft model. They propose SecureSD, which applies stricter verification to early-position draft tokens, improving security while preserving inference efficiency and utility on security and utility benchmarks.

LLM

SecureSD

AI Security· Added 7 Oct 2026, published 6 Oct 2026

arxiv.orgpaper🛡️ Security

Surviving the Router: Optimizing Skill Injections for Retrieval and Execution

This paper studies prompt-injection attacks against AI agent "skill" marketplaces where a router dynamically selects which third-party skill to execute. The authors show existing injection attacks lose 87-97% of their effective attack success rate once a skill must first win retrieval against competing skills, and introduce CORSA (Cluster Optimization for Router-Aware Skill Attacks), which jointly optimizes injections for retrieval and execution, improving end-to-end attack success while preserving user utility and transferring across different router architectures and LLM backbones.

LLM

CORSASkillRouter

AI Security· Added 7 Oct 2026, published 6 Oct 2026

arxiv.orgpaper🛡️ Security

RAISED: Self-Distillation for Robustness to Prompt Injection in LLM Agents

This arXiv paper introduces RAISED (Robust Attack Invariance through Self-Distillation), a training framework that defends tool-using LLM agents against indirect prompt injection without the utility degradation seen in prior training-based defenses. The authors show existing defenses cause distributional drift that makes models wrongly refuse legitimate tool-output-guided steps on benign tasks, then fix this by having a model generate its own tool-use scenarios and self-distill to match clean-context behavior on both clean and injected trajectory variants. Results show reduced attack success rate on injected tool responses while preserving performance on agentic and general-purpose benchmarks.

LLM

RAISED

AI Security· Added 6 Oct 2026, published 5 Oct 2026

arxiv.orgpaper🛡️ Security

COPEX: Benchmarking LLM Robustness to Adversarial Context Across Model Context Protocol Layers

COPEX (COntext Provider EXploitation) is a benchmark that isolates the tool-selecting LLM as the unit of evaluation in Model Context Protocol systems, fixing the surrounding agent stack and varying only the model to measure its intrinsic susceptibility to adversarial context. It covers 25 attack types across 125 scenarios spanning four entry surfaces — model/agent, client, server/tool, and transport — and tests nine models across 3,375 trials, finding a mean attack success rate of 64.4% (58.3%–71.4% by surface); combined input and context scanning cut mean attack success by 49.6% on an eight-attack defense subset. The work separates "system exposure" (attacks succeeding outside the model's observation/control) from "model susceptibility," and the benchmark itself is released publicly.

Model Context ProtocolLLM

COPEX

AI Security· Added 6 Oct 2026, published 3 Oct 2026

arxiv.orgpaper🧠 AI

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

This arXiv paper introduces threat-preserving representation sensitivity (TPRS), a metric for how much an agent-security benchmark's attack success rate (ASR) changes purely from cosmetic changes to tool-name wording, with the underlying task, policy, and ground truth held fixed. Testing on Agent Security Bench (ASB), MCPTox and AgentDojo with GPT-5-mini, Claude Haiku 4.5 and GPT-4o-mini, the authors find swings of up to 13 percentage points in ASR from renaming tools as threat-neutral vs. threat-explicit, showing that single-representation ASR scores are not reliable robustness measures and arguing for evaluation across a controlled set of representations instead.

Agent Security BenchMCPToxAgentDojo+4

evals· Added 5 Oct 2026, published 2 Oct 2026

arxiv.orgpaper🛡️ Security

Threat-Preserving Representation Sensitivity in Agent-Security Benchmarks

This paper introduces threat-preserving representation sensitivity (TPRS), a metric showing that attack success rate (ASR) scores on LLM-agent security benchmarks shift dramatically based on cosmetic changes to tool naming, even when the underlying task, policy, and ground truth are held fixed. Testing on Agent Security Bench, MCPTox, and AgentDojo, the authors find swings of up to ~13 percentage points in ASR on GPT-5-mini and Claude Haiku 4.5 just from renaming tools between threat-related and threat-neutral labels, with a matched-length control reproducing most of the effect. The findings argue that single-representation ASR scores are not reliable robustness claims and that benchmarks should report results across multiple threat-preserving representations.

GPT-4o

Agent Security BenchMCPToxAgentDojo+3

AI Security· Added 5 Oct 2026, published 2 Oct 2026

arxiv.orgpaper🛡️ Security

EvoRiskBench: An Evolving Benchmark for Runtime Security Risks in Workspace Agents

EvoRiskBench is an arXiv paper introducing an evolving benchmark for runtime security risks in LLM-powered workspace agents, built around the EP-Path-EF framework (nine entry-point categories, five effect categories) linking risk entry points to technical effects via agent-mediated paths. The authors ran 450 adversarial tasks across six scenarios against nine model-harness combinations (GPT-5.6 Sol, DeepSeek-V4-Pro-0813, Claude Opus 5 paired with Claude Code, Codex, and OpenClaw), finding Codex + DeepSeek-V4-Pro-0813 the most vulnerable at 68.44% attack success rate and showing ASR varies more by model than by harness. Cases are auto-constructed and verified in isolated environments using runtime traces and environment state.

Claude CodeCodexOpenClaw+1

EvoRiskBenchEP-Path-EFGPT-5.6 Sol+2

AI Security· Added 5 Oct 2026, published 2 Oct 2026

arxiv.orgpaper🛡️ Security

Evaluating and Improving the Robustness of Large Language Models to Input Sequence Variations

An arXiv thesis-style paper proposing R_stab(f), a Jensen-Shannon-divergence-based robustness metric for LLM outputs under small input perturbations, along with a proven bound V(h) <= 1 - R_class(h) for localized attacks. It introduces ASA, an adaptive evolutionary black-box attack against LLM-as-a-Judge systems reaching up to 73.8% attack success rate (62.6% cross-model transfer), evaluates trojan/backdoor detection on the Trojan Detection Challenge 2023 (Pythia-1.4B) and SaTML CTF 2024, shows that committees of 5-7 heterogeneous models cut Gemma-3-4B's ASR to 19.3%, and presents AttestMCP, an HMAC-based tool-call attestation scheme plus a "Commit Boundary" isolation pattern for MCP agents that cuts average ASR on the 847-scenario MCPBench from 53.7% to 12.4%. Methods ship as the JudgeGuard, TrojanArmor, and MCPSec software suites.

Model Context ProtocolLLMGemma+1

Trojan Detection Challenge 2023SaTML CTF 2024MCPBench+7

AI Security· Added 5 Oct 2026, published 1 Oct 2026

arxiv.orgpaper🛡️ Security

The Innocent Courier: Covert Exfiltration Through Legitimate LLM Web Fetching

LLMLeak is an attack that abuses an LLM agent's legitimate web-fetching tool as a covert exfiltration channel for malware that cannot itself reach the internet. A local malicious component embeds a secret in a URL disguised as useful reference material (e.g. for a library migration task); when the LLM fetches it, the secret reaches an attacker-controlled DNS or web server, achieving a 79.7% attack success rate across eleven open-parameter models and working against real-world chatbots.

LLM

LLMLeak

AI Security· Added 2 Oct 2026, published 1 Oct 2026

decrypt.coarticle🧠 AI

Gemini 4 Is Here, and Google’s Flagship Tops All Other AI Models on Cybersecurity

Google released Gemini 4 Argon, its new frontier model for coding, office work and cyber defense, scoring 77.9% on DeepSWE v1.1 (vs. 74.2% Claude Opus 5.5, 74.1% GPT-6 Astra) and expanding max output to 1 million tokens. On Gray Swan's indirect prompt injection benchmark it posted a 0.7% attack success rate, edging out Claude Opus 5.5 and Claude Fable 5.1 (both 1.0%), and scored 70.9% on Google's internal Wiz Penetration Test Benchmark versus 58.2% for the earlier restricted Gemini 3.8 Flash Cyber. The model initially ships without cyber guardrails only to vetted defenders via Google's Fairwind Program before wider API and Google AI Ultra release at $2/$10 per million input/output tokens.

Gemini

GoogleClaude Opus 5.5Claude Fable 5.1+16

models, evals· Published and added 30 Sep 2026

decrypt.coarticle🛡️ Security

Gemini 4 Is Here, and Google’s Flagship Tops All Other AI Models on Cybersecurity

Decrypt covers Google's Gemini 4 Argon launch, reporting it scored a 0.7% attack success rate on Gray Swan's Indirect Prompt Injection benchmark, beating Claude Opus 5.5 and Claude Fable 5.1 (both 1.0%), GPT-6 Astra (8.5%), and far outperforming Grok 4.6 and Kimi K3 (~52% each). It also details Argon's cyber-defense capability: it ships first to vetted defenders through Google's Fairwind Program "without cyber guardrails," scored 70.9% on the internal Wiz Penetration Test Benchmark writing working exploits against real web-app flaws, and reportedly helped Wiz find a critical healthcare-software flaw earlier models missed.

GeminiClaudeOpenAI+3

Fairwind ProgramGray SwanGemini 4 Argon+8

AI Security· Published and added 30 Sep 2026

arxiv.orgpaper🛡️ Security

Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreaks

Researchers report that attaching an unrelated "decoy" image to an encoded jailbreak prompt can sharply cut attack success rate against caption-mediated black-box VLM defenses like ECSO — up to 73 percentage points — even though the image itself is content-free, across five frontier VLMs, two encoded-attack families, and three defenses. Controls show the effect stems from image presence (not content or vendor filtering) and holds even under adaptive attacks targeting the caption re-check, but unconditional decoy attachment inflates benign refusal rates by 10–67pp, so the authors propose gating it on a lightweight encoded-input detector. The paper was withdrawn by the author after v2, so treat conclusions as unverified/retracted.

ECSO

AI Security· Added 26 Sep 2026

arxiv.orgpaper🛡️ Security

MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers

MCPTox is a benchmark for evaluating Tool Poisoning Attacks against LLM agents using the Model Context Protocol, where malicious instructions are hidden in a tool's metadata rather than its output. Built on 45 live real-world MCP servers and 353 authentic tools, it generates 1312 malicious test cases across 10 risk categories via three attack templates, and finds widespread vulnerability across 20 LLM agents: o1-mini has a 72.8% attack success rate, more capable models are often more susceptible due to stronger instruction-following, and even the best model (Claude-3.7-Sonnet) refuses less than 3% of attacks.

Model Context Protocolo1-miniClaude-3.7-Sonnet

Tool Poisoning AttackMCPTox

AI Security· Added 26 Sep 2026

arxiv.orgpaper🛡️ Security

Region-Level Black-Box Defense Against Stealthy Embedding-Space Backdoors in CLIP

CLIPGuard is a fully black-box defense against embedding-space backdoor attacks in CLIP, which poison a tiny fraction of image-text pairs to implant stealthy triggers that shift the joint embedding space without needing model parameters, gradients, or clean validation data. It detects malicious image regions by measuring segment-wise embedding perturbations and purifies only suspicious segments via semantic inpainting, reducing attack success rate to 1.05% while keeping clean accuracy at 86.34% on STL-10 and ImageNet against BadCLIP, BadNets, blended, patch-based, and typographic triggers, outperforming CleanCLIP and CleanerCLIP.

CLIPCLIPGuard

BadCLIPBadNetsCleanCLIP+3

AI Security· Added 28 Sep 2026, published 25 Sep 2026

arxiv.orgpaper🛡️ Security

A2M: Trace-Optimized Agent Hijacking in the MCP Ecosystem

A2M (Attraction-to-Manipulation) is a two-stage black-box framework for hijacking agents that use the Model Context Protocol to select tools from third-party MCP servers. The Attraction phase optimizes attacker-controlled tool metadata to raise invocation probability, while the Manipulation phase uses execution traces to craft adversarial tool outputs that steer agent behavior; on LiveMCPBench it achieves a 93.6% malicious tool invocation rate against GLM-4.6, inflates token cost 32.4x under a cognitive denial-of-service variant, and reaches a 74.4% mean attack success rate across information exfiltration, environment integrity compromise, and reasoning derailment, with partial transfer (63.6%/2.7x/24.5%) to four other models without re-optimization. The authors argue this exposes a semantic supply-chain risk in MCP tool ecosystems and call for stronger tool vetting and runtime isolation.

Model Context ProtocolGLM

LiveMCPBenchA2MCognitive Denial of Service

AI Security· Added 2 Oct 2026, published 22 Sep 2026

arxiv.orgpaper🛡️ Security

ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack

ReasAlign is a model-level safety alignment technique that defends LLM agents against indirect prompt injection by adding structured reasoning steps to detect conflicting instructions embedded in external data and preserve the user's original task intent. It also uses a test-time scaling mechanism with a preference-optimized judge model to score and select the best reasoning trajectory. On the CyberSecEval2 benchmark it reports 94.6% utility with only 3.6% attack success rate, substantially outperforming Meta SecAlign, which scores 56.4% utility with a 74.4% attack success rate.

LLM

Meta SecAlignCyberSecEval2

AI Security· Added 22 Sep 2026

arxiv.orgpaper🛡️ Security

PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defenses

PISmith trains an attack LLM with GRPO in a black-box setting to generate injected prompts that defeat prompt-injection defenses. It addresses extreme reward sparsity, which causes entropy collapse in standard GRPO, with adaptive entropy regularization and dynamic advantage weighting. Evaluated on 13 benchmarks against 8 defenses and 7 baselines (static, search-based such as TAP and PAIR, and RL-Hammer), it achieves the highest attack success rates and also works on InjecAgent and AgentDojo against GPT-4o-mini and GPT-5-nano. It shows state-of-the-art defenses such as Meta-SecAlign remain vulnerable to adaptive attacks.

LLMAgentDojo

PISmithRL-HammerTAP+8

AI Security· Added 17 Sep 2026

deploymentsafety.openai.comwriteup🛡️ Security

GPT-6 Astra System Card

OpenAI's system card for GPT-6 Astra, the first model to reach the "Critical" cybersecurity capability level under OpenAI's Preparedness Framework, meaning it can autonomously find and exploit unknown vulnerabilities in well-protected systems. The card details jailbreak and indirect prompt-injection robustness testing (including an external Gray Swan IPI Arena benchmark showing an 8.5% attack success rate vs. 27.0% for GPT-5.6 Sol), a decline in chain-of-thought monitorability with evidence the model can evade monitors via sandbagging under adversarial conditions, and a new "honeypot" alignment evaluation built on ExploitGym capture-the-flag tasks that was developed in response to the prior Hugging Face incident. It also covers internal deployment safeguards (checkpoint encryption, universal CoT monitoring, blocking alignment evaluations) and introduces "verbalized metagaming" and "oversight gaming" as alignment-evasion concepts. Note: fetched content is truncated to the first ~40,000 of ~222,000 characters, so later sections (deployment simulations, further alignment details) are not reflected here.

OpenAIHugging FaceCodex+1

CVE-2023-6702CVE-2024-0517GPT-6 Astra+9

AI Security· Added 7 Oct 2026, published 3 Sep 2026

github.comrepo🛡️ Security

henrymao2004/Auto-research-red-teaming

AHA (Agent Hacks Agent) is an autonomous red-teaming framework where a researcher agent iteratively hypothesizes attacks, runs them against Docker-sandboxed victim agents (built on Claude Code and Codex, targeting frontier models like Minimax-M2.7, Kimi-K2.6, and Deepseek-V4-Pro), and accumulates a persistent Vulnerability Concept Graph of reusable attack mechanisms rather than one-off jailbreak strings. It reports held-out Attack Success Rate evaluations showing these mechanisms — notably a "claimed-authorization" bypass pattern — transfer across scenarios and victim models (up to ~88% of native ASR), and ships as registry-discovered plugins for scenarios like AgentHazard, AgentDojo-style indirect injection, and DecodingTrust-Agent.

Claude CodeCodexDocker+2

AgentHazardAgentDojoDecodingTrust-Agent+2

AI Security, Offensive Security· Published and added 25 Aug 2026

trendmicro.comwriteup🛡️ Security

Sockpuppeting: How a Single Line Can Bypass LLM Safety Guardrails | TrendAI (US)

Trend Micro details sockpuppeting, a jailbreak technique that injects a fake acceptance prefix such as Sure here is as the final assistant-role message via the API assistant-prefill feature, exploiting LLM self-consistency to bypass safety training with no gradient access or optimization required. The team tested 14 prefix variants across 11 models from four providers including GPT-4o, Claude 4 Sonnet, Gemini 2.5 Flash, Qwen, Llama, and Gemma, finding every model that accepted the prefill was at least partially vulnerable, with Gemini 2.5 Flash worst at 15.7 percent attack success rate, while OpenAI, AWS Bedrock, and Claude 4.6 block the attack entirely by rejecting assistant-terminated message ordering. Self-hosted inference servers like Ollama and vLLM do not enforce this validation by default, leaving self-hosted deployments exposed.

GPT-4oClaudeGemini+6

sockpuppetingGreedy Coordinate GradientDo Anything Now+4

AI Security· Added 16 Aug 2026

github.comrepo🛡️ Security

autoguardrails

autoguardrails is an open-source alignment research scaffold from Santander AI Lab, inspired by Karpathy's autoresearch pattern, that searches over a single mutable policy.md file to minimize attack success rate (ASR) against a fixed jailbreak/obfuscation eval suite while enforcing a benign-pass floor so the guardrail can't just refuse everything. It reports asr_unguarded, asr_with_policy, and policy_delta to isolate the guardrail's contribution from the target model's native safety, and supports both an offline deterministic stub and real OpenAI-compatible target/judge models.

PythonOpenAI

AI Security· Published and added 5 Aug 2026

github.comrepo🛡️ Security

Microsoft Agent Governance Toolkit

Agent Governance Toolkit (AGT) is a Microsoft open-source framework that intercepts every tool call, message, and delegation made by an AI agent in deterministic application-layer code before it reaches the model, enforcing YAML/OPA/Cedar policies, agent identity (SPIFFE/DID/mTLS), and tamper-evident audit logs. It ships SDKs for Python, TypeScript, .NET, Rust, and Go, a Claude Code plugin, and CLI tools (agt verify, agt red-team scan) for OWASP-style compliance checks and prompt-injection audits. The project explicitly frames prompt-level safety as insufficient, citing a 100% adversarial attack success rate on GPT-4o/Claude 3/Llama-3 (Andriushchenko et al., ICLR 2025) as motivation for application-layer enforcement instead.

Model Context ProtocolClaude CodePython+3

OWASP LLM01:2025JailbreakBench

AI Security· Published and added 6 May 2026