49 results for “Prompt Injection”

claude-skills is an open-source MIT library of about 398 Claude Code skills, agents and plugins that also installs into Codex, Gemini CLI, Cursor and 9 other coding agents. The security-relevant parts are skill-security-auditor, a stdlib-only Python scanner that checks skills for command injection, data exfiltration, prompt injection and dependency supply-chain risks before install, plus a security-guidance PreToolUse hook, a security suite and compliance skills (ISO 27001, SOC 2, GDPR).

github.comrepoAdded 25 Oct 2026🛡️ Security

Arcjet JS SDK is an Apache-2.0 JavaScript monorepo that adds runtime security checks to AI apps, agents and HTTP endpoints. It ships framework SDKs (Next.js, Node.js, Bun, Deno, Fastify, NestJS) plus a Guard API for agent tool calls, MCP handlers, queue workers and background jobs that have no HTTP request object. Features include prompt injection detection, sensitive-information detection (PII, credit cards, custom patterns) and token-bucket, fixed-window and sliding-window rate limiting.

x.comrepoPublished and added 11 Oct 2026🛡️ Security

The Vulnerable MCP Project is a curated, continuously growing database of security vulnerabilities and attack techniques specific to the Model Context Protocol ecosystem. Entries cover concrete CVEs (e.g. CVE-2025-68143/68144/68145 in Anthropic's mcp-server-git), transport-layer SDK flaws (cross-client response leakage, missing DNS rebinding protection in the official TypeScript/Python SDKs), unauthenticated/0.0.0.0-bound servers enabling RCE (MCPJam inspector, MCP Inspector, Grafana MCP), command/path injection in dozens of third-party servers (gemini-mcp-tool, mcp-server-kubernetes, Framelink Figma MCP, Zen MCP, filesystem-mcp), SSRF chains reaching cloud metadata (MarkItDown MCP, mcp-fetch-server), and a large set of AI-specific attacks: tool poisoning, trigger-phrase data exfiltration, prompt injection via GitHub issues, Jira and Google Docs, rug-pull tool mutation, and cross-server tool-name collisions. It functions as the de facto vulnerability registry for the MCP ecosystem, useful for tracking disclosed issues before patching or adopting third-party MCP servers.

vulnerablemcp.inforepoAdded 6 Oct 2026🛡️ Security

Release notes for garak v0.17.0, NVIDIA's open-source LLM vulnerability scanner used for red-teaming and probing models for jailbreaks, prompt injection, data leakage and other failure modes. The release adds an EU AI Act risk-category mapping to tag and group probe results, plus fixes to detectors (agent_breaker judge verdict validation, packagehallucination import parsing, MarkdownExfilContent divide-by-zero, TriggerListDetector alignment) and probes (badchars ASCII selection, Markdown exfiltration link templates), along with Ollama generator improvements and dropped Python 3.10 support in favor of 3.13.

github.comrepoAdded 29 Sep 2026🛡️ Security

security-harness is an open-source Claude Code plugin (also packaged for Gemini CLI, Codex and opencode) that orchestrates subagents — sh-recon, sh-hunter, sh-chainer, sh-verifier, sh-reporter — through a static-analysis pipeline covering 15 vulnerability classes (IDOR/BOLA, SQLi, XSS, SSRF, injection, auth, deserialization, path traversal, secrets, CSRF, XXE, open redirect, crypto, race conditions, file upload), producing README/JSON/SARIF/HTML/PDF reports with PoCs, CVSS and CWE/OWASP mappings. It also ships sh-pr-review, a per-PR merge-gate skill that posts inline findings and a commit status scoped only to code the PR itself introduced or aggravated, with caching, dedup via fingerprints, and a scheduled GitHub Action that keeps its per-class knowledge bases fresh by having one AI agent draft updates from allowlisted sources (OWASP, PortSwigger Research, CWE/CAPEC, HackerOne) while a separate adversarial reviewer agent fails-closed against prompt injection before auto-merge.

github.comrepoAdded 2 Oct 2026, published 29 Sep 2026🛡️ Security

garak is NVIDIA's open-source LLM vulnerability scanner that runs probes/detectors for jailbreaks, prompt injection, PII leakage, package hallucination, markdown exfiltration and more against LLM targets. This v0.17.0 release adds a new EU AI Act risk-category mapping to tag and group probe results by the Act's risk categories, alongside fixes to the agent_breaker judge, markdown-exfiltration detectors, Ollama auth/generation support, and dropped Python 3.10 in favor of 3.13 support.

github.comrepoAdded 26 Sep 2026🛡️ Security

Agent Threat Rules (ATR) is an open, versioned YAML rule format (ATR-YYYY-NNNNN scheme) for detecting AI-agent runtime threats — prompt injection, tool-poisoning, malicious SKILL.md content, and MCP exchange abuse — evaluated by a reference TypeScript engine (with a Python wrapper) shipped as an npm/pip package, CLI, GitHub Action, and Docker scanner with SARIF output. The project positions itself as "Sigma for agent threats," maps rules to OWASP Agentic/LLM Top 10, MITRE ATLAS, and NIST AI RMF, and publishes an extensive (and unusually self-critical) benchmark suite against corpora like garak, PromptBench, and PromptInject, including documented harness bugs and withdrawn recall figures.

github.comrepoAdded 26 Sep 2026🛡️ Security

Release changelog for MITRE ATLAS's atlas-data repository, the structured YAML/STIX dataset behind the ATLAS knowledge base of adversary tactics and techniques targeting AI-enabled systems (prompt injection, model/data poisoning, AI agent tool abuse, AI supply chain compromise). The v2026.09 release brings the matrix to 120 techniques, 88 sub-techniques, 40 mitigations, and 73 case studies, and recent versions introduced a new v6 YAML data format with Pydantic/SQLAlchemy/FastAPI tooling, platform tagging (Predictive AI, Generative AI, Agentic AI, Enterprise), and renamed/generalized techniques like "AI Agent Tool Poisoning," "Manipulate AI Model," and "Enterprise Resource Discovery." It's the canonical, actively-maintained taxonomy practitioners use for AI red-teaming, threat modeling, and mapping real-world AI attack case studies.

github.comrepoAdded 26 Sep 2026🛡️ Security

who-let-the-agents-act is a self-hostable lab suite (Docker/Python, Groq-backed LLM agent) with nine isolated scenarios — overpowered data tools, IDOR-style cross-account access, unsafe agent handoffs, refund-limit bypass, indirect prompt injection via poisoned invoices, multi-tenant RAG leakage, secret leakage through debugging, fail-open approval outages, and a multi-hop confused-deputy agent chain. Each scenario runs in multiple modes (vulnerable vs. application-enforced) against the same prompt, recording model plan, policy decision, tool calls, and DB access as JSON so a reader can directly compare what changes when authorization is enforced outside the system prompt.

github.comrepoAdded 26 Sep 2026🛡️ Security

DVLAA (Damn Vulnerable LLM and Agent Application) is a self-hosted, Docker-deployable security training range for LLM and agent applications, modeled on the DVWA concept but for AI. It covers OWASP LLM Top 10 and Agent Security Top 10 risks (prompt injection, RAG poisoning, tool abuse, memory poisoning, excessive resource consumption, etc.) through 81 hands-on challenges, including an "AWDP" attack/defense track where 10 challenges reproduce real disclosed vulnerabilities in Dify, RAGFlow, Langflow, Flowise, Open WebUI and n8n (with optional real upstream container reproduction), 20 challenges adapted from a China–ASEAN AI security CTF, and 6 challenges adapted from a real model-poisoning CTF. It provides real model interaction, state-machine flag verification, source/prompt viewers, model management (local, Ollama, SiliconFlow, OpenAI-compatible), and bilingual UI.

github.comrepoAdded 22 Sep 2026🛡️ Security

jev-guard is an MIT-licensed hook layer that adds an auto mode to Claude Code, Codex, Copilot CLI, Gemini CLI, Cursor, pi, OpenCode, Antigravity and ACP editors. It asks the Jev model three typed questions (risk, user_requested, from_untrusted) per tool call and decides deny, ask or allow in about 20 lines of code. It also scans tool results for prompt injection and checks skills and plugins.

github.comrepoAdded 27 Oct 2026, published 17 Sep 2026🧠 AI

A GitHub-hosted, CC0-licensed index of 281 public writeups on attacks against AI agents, organized into six categories: indirect prompt injection, direct prompt injection/jailbreaks/guardrail bypass, MCP and agent tooling, AI platform/infrastructure bugs, classic web bugs in AI product surfaces, and AI used as an attacker's tool. Entries span 2021-2026 and include major disclosures like GeminiJack, ForcedLeak, Clinejection, CVE-2025-53773 (GitHub Copilot RCE), and dozens of Johann Rehberger/PromptArmor/qual1a writeups covering exfiltration, MCP OAuth flaws, and coding-agent sandbox escapes. Comes with an accompanying blog post "How to Hack AI Agents" and a scripted links.txt for automation.

github.comrepoAdded 22 Sep 2026, published 17 Sep 2026🛡️ Security

Agent Egress Bench is an open, tool-neutral benchmark maintained by PipeLab (makers of Pipelock) that tests whether products mediating an AI agent's outbound traffic — proxies, egress firewalls, MCP wrappers, agent gateways — actually catch exfiltration attempts caused by prompt injection or misalignment, rather than testing the underlying model itself (that's left to AgentDojo, InjecAgent, CyberSecEval, AgentHarm, ASB). It ships a reproducible Go runner with a proxy/WebSocket/MCP/A2A adapter contract, an offline validator, DSSE-signed evidence bundles, and a CI-runnable GitHub Action, and separates containment from false-positive rate as two distinct scored axes with immutable case IDs and OWASP Agentic Top 10/MITRE ATT&CK mappings. Its governance explicitly permits third parties to publish results unfavorable to the maintainer's own Pipelock product.

github.comrepoAdded 12 Sep 2026🛡️ Security

A curated GitHub corpus tracking real-world security incidents, CVEs, attack techniques, and defensive tools specific to autonomous AI agents — spanning prompt injection/goal hijacking, MCP client-side tool-metadata poisoning, the "Promptware Kill Chain" multi-stage attack model, MITRE ATLAS agentic techniques, and the OWASP Top 10 for Agentic Applications. It also documents the OpenClaw/ClawHub supply-chain incident (1,184 malicious skills confirmed by Antiy CERT) and links defensive projects like OWASP Agent Memory Guard for LangChain/LlamaIndex agent memory protection.

github.comrepoAdded 3 Sep 2026🛡️ Security

galyarder-security (formerly Shannon Proxy) is a Docker/Temporal-based auditing layer that intercepts model SDK calls (Anthropic, Groq, local Ollama, etc.) and routes them through a local proxy on port 8317 rather than direct cloud endpoints, giving operators visibility into and control over what autonomous coding/tool-using agents execute. It runs structured "security sweep" and threat-modeling workflows against a target repo (`./shannon start`) and includes deterministic cleanup routines (`./shannon cleanup`) to remove agent-generated scripts and execution residue and prevent workspace pollution/exposure. It targets a threat model broader than chat-based prompt injection: agent tool-execution drift, dependency supply-chain issues, and unverified code paths in "agentic company" shell-access workflows.

github.comrepoPublished and added 31 Aug 2026🛡️ Security

A curated GitHub awesome-list cataloging the emerging agentic-AI security ecosystem: dozens of Model Context Protocol servers that give LLM agents access to security tools (Nuclei, Shodan, VirusTotal, Burp Suite, Ghidra, BloodHound, x64dbg), autonomous penetration-testing and CTF-solving agent frameworks (CAI, pentagi, BreachSeek, Dark-Moon, D-CIPHER), scanners/guardrails that audit agent skills, MCP servers, and multi-agent chains for prompt injection and other risks (Aguara, Inkog, MCP Doctor, AI-Infra-Guard, ATR), agentic SOC/detection platforms (Vigil), threat-modeling frameworks for agentic systems (MAESTRO, ATFAA/SHIELD), research papers, benchmark datasets, and community/learning resources. It functions as a living index for anyone building or defending AI-agent-driven security tooling.

github.comrepoAdded 3 Sep 2026, published 31 Aug 2026🛡️ Security

DarkMoon is a self-hosted, GPLv3 AI penetration testing platform that orchestrates 50 specialist agents over an MCP-gated toolbox (Nuclei, BloodHound, sqlmap, Impacket, Naabu, httpx, ffuf, etc.) to autonomously run recon-to-exploitation across web, API, Active Directory, Kubernetes, multi-cloud, CI/CD, databases, IoT firmware, and AI/LLM inference endpoints. It uses a Privacy Gateway that tokenizes real IPs/hosts/credentials into placeholders so raw infrastructure data never reaches the (optionally local, e.g. Ollama/llama.cpp) LLM, and can generate sandbox-validated, exploit-retested pull requests for findings. It also ships a dedicated LLM agent that probes OpenAI-compatible/Ollama/vLLM/TGI inference endpoints for the OWASP LLM Top 10 (prompt injection, jailbreaks, system-prompt leakage) using garak-backed probes.

github.comrepoPublished and added 5 Aug 2026🛡️ Security

SecureAI-Scan is an offline CLI that performs import-resolved dataflow analysis on TypeScript, JavaScript, and Python code to find LLM/MCP/Agent-Skill/RAG-specific risks (prompt injection, tool poisoning, unsafe output handling, vector-store access control), mapped to the OWASP Top 10 for LLM Applications, Agentic Applications, and MCP Top 10. It distinguishes proven dataflow traces from heuristic keyword matches, scans .mcp.json/claude_desktop_config.json/SKILL.md for tool-poisoning and command-injection patterns (citing the WhatsApp MCP rug-pull and postmark-mcp backdoor), and includes evasion-resistant deobfuscation checks built in response to the "Cloak and Detonate" adversarial study (arXiv:2607.02357) that bypassed most published skill scanners. It also runs offline advisory checks against known-malicious/vulnerable packages (validated against CVE-2025-6514 and CVE-2024-34359) and integrates via GitHub Actions, pre-commit hooks, SARIF output, and an MCP server for Claude Code.

github.comrepoPublished and added 5 Aug 2026🛡️ Security

ClawSecCheck audits OpenClaw agent installs for prompt injection, malicious MCP servers, and supply-chain risk in skills and dependencies.

github.comrepoPublished and added 22 Jul 2026🛡️ Security

Yolorouter is a free, self-hosted, single-binary LLM gateway that translates between the four major chat wire protocols — OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, and Gemini generateContent — so a client built for one provider (e.g. Claude Code) can be pointed at any other. It adds multi-provider failover, round-robin upstream API key pooling with rate-limit/quota handling, per-key budgets and access control, a multi-user admin console with OAuth2/OIDC SSO, and native image/video generation support for OpenAI-shaped and non-OpenAI-shaped providers (DashScope, Volcengine Ark, Kling, MiniMax). It also ships cost-optimization features — deterministic system-prompt injection and tool-output compression for coding agents — backed by a published benchmark, plus built-in usage/cost observability.

github.comrepoAdded 2 Oct 2026, published 21 Jul 2026🧠 AI

SingGuard-NSFA is an open-source guardrail framework (released as 0.8B/2B/4B/9B model checkpoints) that targets operational threats in LLM agents—prompt injection, sensitive-info extraction, malicious code requests, tool misuse, and resource exhaustion—rather than just content-safety issues. It's built on a CIA-triad-grounded taxonomy of 185 risk variants across 7 domains (cross-validated against OWASP guidelines) and a 133-language benchmark with 93K+ samples, offering both generative chain-of-thought auditing and ~50ms real-time classification heads. It beats Llama Guard 3, WildGuard, and GPT-5.1 by 6-12 F1 points on multilingual benchmarks, and can also be plugged into Llama Guard 3 as a lightweight enhancement (+17.6 F1).

github.comrepoPublished and added 19 Jul 2026🛡️ Security

LLMVault is a self-hosted, deliberately-vulnerable training range modeled on WebGoat but for LLM applications, covering 25 labs across the OWASP Top 10 for LLM Applications (2025) - prompt injection, jailbreaking, data/model poisoning, agent exploitation, model extraction, and insecure downstream output handling. It ships two modes: a scored Play Mode with scripted assistants for repeatable flag capture, and an unscripted Live Mode that points the same attacks at a real local Ollama model (qwen2.5:3b-instruct) with a freshly minted per-session secret, requiring the player to actually talk the model into leaking it. Each lab pairs the attack with a written defense explanation, and an expert tier is shipped AES/Fernet-encrypted and unlocked manually by the maintainer.

github.comrepoPublished and added 18 Jul 2026🛡️ Security

HOL Guard is an open-source local "antivirus" runtime protection layer for AI coding agents that intercepts shell commands, file access, package installs, and MCP tool calls before execution, classifying them for secret exposure, destructive operations, prompt injection, and supply-chain risk, then allowing/blocking/requesting approval per policy. It integrates via native hooks and managed MCP proxies with Codex, Claude Code, GitHub Copilot CLI, Cursor, Cline, Gemini CLI, Grok, and other agents, and includes a supply-chain package/advisory checker (hol-guard supply-chain scan/explain). The repo also bundles plugin-scanner, a companion CLI/GitHub Action that lints and scores agent plugins, skills, and MCP integrations for manifest issues, secrets, MCP transport risks, and dependency/lockfile problems, producing SARIF/JSON reports for CI gating.

github.comrepoPublished and added 15 Jul 2026🛡️ Security

LLM Guard (Protect AI) is an open source Python toolkit that wraps input/output scanners around LLM applications to detect and block prompt injection, PII/data leakage, toxic or harmful content, and other unsafe interactions, with a pip-installable library and API deployment option. The project has since been archived and is no longer maintained, though the code and associated Hugging Face models remain publicly available.

github.comrepoAdded 3 Sep 2026, published 14 Jul 2026🛡️ Security

promptfoo is an open-source CLI and library for evaluating and red-teaming LLM applications, letting developers run automated evals, compare model providers side-by-side, and scan for LLM-specific vulnerabilities like jailbreaks and prompt injection. It runs entirely locally, integrates into CI/CD for automated checks, and can generate security vulnerability reports and review pull requests for LLM-related security and compliance issues. The project was recently acquired by OpenAI but remains MIT-licensed and open source.

github.comrepoAdded 3 Sep 2026, published 14 Jul 2026🛡️ Security

GitHub repo by Kai Greshake collecting research and proof-of-concept material on LLM security, notably indirect prompt injection attacks where malicious instructions are embedded in retrieved content (web pages, documents) to hijack LLM-integrated applications like Bing Chat. It underpins some of the earliest published demonstrations of prompt injection as a practical attack vector against production AI assistants.

github.comrepoAdded 3 Sep 2026, published 14 Jul 2026🛡️ Security

Awesome LLM Security is a GitHub-curated list of papers, tools, and datasets covering LLM attack and defense research, including jailbreak techniques (GCG, AutoDAN, DeepInception, GPTFUZZER, cipher-based jailbreaks), indirect prompt injection, backdoor/virtual prompt injection attacks, model fingerprinting (LLMmap, TRAP), and defenses like PARDEN, circuit breakers, and self-defense filtering. It's a comprehensive reference for tracking the state of adversarial and defensive LLM security research from 2022 onward.

github.comrepoAdded 3 Sep 2026, published 14 Jul 2026🛡️ Security

garak is NVIDIA's open-source LLM vulnerability scanner, described as an nmap/Metasploit for LLMs that probes generative models for hallucination, data leakage, prompt injection, jailbreaks, toxicity, and other failure modes. It supports a wide range of targets such as Hugging Face, OpenAI, AWS Bedrock, Cohere, Groq, llama.cpp/gguf, REST endpoints, and NIM, and structures scans around pluggable probes, detectors, harnesses, and generators, producing JSONL reports and hit logs.

github.comrepoPublished and added 14 Jul 2026🛡️ Security

Giskard is an open-source Python testing/evaluation library for agentic and LLM systems, rewritten in v3 as a modular set of packages (giskard-checks, giskard-scan, giskard-core, giskard-llm, giskard-agents). giskard-scan is the red-teaming layer: it auto-generates adversarial test suites from a plain-language agent description, covering prompt injection, harmful content, stereotypes, and misinformation across OWASP LLM Top-10 categories, and includes a built-in prompt-injection payload dataset. giskard-checks provides LLM-as-judge and assertion-based evals (Groundedness, Conformity) for catching regressions and enforcing safety rules in multi-turn agent conversations; the legacy v2 also offered a tabular ML bias/robustness scanner and RAGET for RAG test-set generation.

github.comrepoPublished and added 14 Jul 2026🛡️ Security

NeMo Guardrails is NVIDIAs open-source Python toolkit (nemoguardrails) for adding programmable rails between an application and an LLM, covering input, dialog, retrieval, execution, and output stages. It ships built-in defenses against jailbreaks and prompt injection, self-check fact-checking and hallucination detection, sensitive-data masking, integrations with NVIDIA safety models, ActiveFence and PolicyAI moderation, LangChain, and a Colang DSL for defining conversational flow rails, plus a CLI/server and an evaluation tool for vulnerability scanning against sample bots.

github.comrepoPublished and added 14 Jul 2026🛡️ Security

Purple Llama is Meta's umbrella repo of tools and evals for building safely with generative AI, bundling Llama Guard (input/output content moderation fine-tuned on Llama 3.1/3.2), Prompt Guard (detects prompt injection and jailbreak attempts), Code Shield (inference-time filtering of insecure LLM-generated code and code-interpreter abuse prevention), and the CyberSec Eval benchmark suite (CWE/MITRE ATT&CK-based tests now spanning insecure code generation, offensive cyber capability, spear-phishing, and visual prompt injection). It matters as one of the first industry-wide, MIT-licensed benchmark and safeguard sets specifically for LLM cybersecurity risk.

github.comrepoPublished and added 14 Jul 2026🛡️ Security

HackAgent is an open-source Python SDK and CLI for testing the security of AI agents, automating research-backed attacks (e.g. jailbreaks, prompt injection) to surface vulnerabilities before exploitation. It runs locally with no API key required (SQLite-backed results and a built-in dashboard), supports an optional cloud reporting mode, and can drive browser-based agents via Playwright for WEB-provider targets. The project is Apache-2.0 licensed and framed explicitly for authorized security testing and coordinated disclosure.

github.comrepoPublished and added 10 Jul 2026🛡️ Security

A community-maintained GitHub guide (requie/AI-Red-Teaming-Guide) that consolidates AI red-teaming methodology, mapping techniques like prompt injection, jailbreaking (Skeleton Key, Crescendo), model extraction, data/fine-tuning poisoning, and MCP tool-poisoning/rug-pull attacks to frameworks including NIST AI RMF, OWASP Top 10 for LLM Applications and Agentic Applications, MITRE ATLAS, and CSA's Agentic AI Red Teaming Guide. It cites real incidents (OpenClaw's CVE-2026-25253, GitHub Copilot's CVE-2025-53773, the Anthropic-disrupted Claude Code state-sponsored campaign, the Morris II AI worm) and surveys red-team tooling (PyRIT, NVIDIA's Garak, DeepTeam) plus emerging agent-orchestrated red-teaming approaches.

github.comrepoPublished and added 8 Jul 2026🛡️ Security

api-relay-audit is a zero-dependency, locally-run Python security audit tool for AI API relays and LLM proxies that sit between a user and providers like Anthropic or OpenAI. It runs a 14-step check covering hidden prompt injection/extraction, context truncation, model-identity substitution, tool-call/package-command tampering, and SSE stream integrity, producing a reviewable Markdown report with LOW/MEDIUM/HIGH findings (inconclusive results stay visible rather than being marked clean). It also adds a web3 profile for wallet-safety probes (transfer guidance, signed-transaction refusal, private-key refusal) and is distributed as a standalone script, a DeepSeek Harness (DSH) plugin, and OpenClaw/Hermes agent skills.

github.comrepoPublished and added 4 Jul 2026🛡️ Security

SkillsGuard is a zero-dependency static security scanner for AI agent skill packages (SKILL.md files and bundled scripts) that detects prompt injection, exfiltration, obfuscated payloads, persistence, and privilege escalation before an agent reads or executes them. It recursively decodes base64/hex/URL-encoded blobs to catch hidden attacks like a base64-wrapped reverse shell, computes a 0-100 risk score, and integrates into CI (SARIF/JSON output), git pre-commit hooks, and Claude via an MCP server (scan_skill tool) or a free hosted Cloudflare Workers API. The README also situates it against competing agent-skill scanners (NVIDIA, Cisco, Snyk, Mondoo, SkillSpector, SkillScan), noting it's the only one requiring no account/API/LLM dependency versus SkillSpector's LLM-augmented semantic analysis backed by a study of 42,447 scanned skills.

github.comrepoPublished and added 21 Jun 2026🛡️ Security

SuperClaw is a pre-deployment security testing framework for autonomous AI coding agents like OpenClaw, generating scenario-driven adversarial tests for prompt injection, jailbreaks, tool misuse, and behavioral drift. It scores agent behavior against explicit security contracts with rubric-based evidence (tool calls, outputs) and produces HTML/JSON/SARIF reports, with local-only mode and auth-token gating for remote targets to reduce misuse risk.

github.comrepoPublished and added 18 Jun 2026🛡️ Security

VICE is an open-source (MIT) Node.js security auditing CLI with two modes: a remote black-box scan that crawls a target with a headless browser across 15-16 security modules (headers, TLS, Supabase access controls, subdomains, storage buckets, WebSockets), and a local white-box audit that scans source code for secrets, .env exposure, npm vulnerabilities, missing Supabase RLS policies, SQL injection and XSS. It ships as a GitHub Action that posts PR comments with a security score, uploads SARIF findings to GitHub Code Scanning, gates merges on a minimum score, and maintains a live README badge. Version 3.4.0 adds an AI/RAG security module for testing chat/agent API endpoints for access control, prompt injection, retrieval isolation, and tool-security issues.

github.comrepoPublished and added 14 Jun 2026🛡️ Security

Pipelock is an open-source (Apache 2.0 core) AI agent firewall that sits between agents and the network, mediating HTTP, WebSocket, MCP, and A2A traffic to detect secret exfiltration, prompt injection, SSRF, and MCP tool poisoning. It scans request/response bodies with a DLP engine (65 built-in patterns, checksum validation), streaming SSE inspection, and MCP tool-call chain analysis, then emits Ed25519-signed, hash-chained action receipts so decisions can be verified offline outside the agent runtime. It also offers OS-level containment (Landlock/seccomp/sandbox-exec), a kill switch, canary tokens, and integrations for Claude Code, Cursor, VS Code, OpenAI Codex, LangGraph, CrewAI, and other agent frameworks.

github.comrepoPublished and added 14 Jun 2026🛡️ Security

SkillSpector is NVIDIA's open-source scanner for AI agent skills (used by Claude Code, Codex CLI, Gemini CLI, etc.), detecting 71 vulnerability patterns across 17 categories including prompt injection, data exfiltration, privilege escalation, MCP tool poisoning, and supply-chain issues via static analysis, AST checks, YARA rules, and optional LLM semantic evaluation. It cites research showing 26.1% of a 31,132-skill sample contain vulnerabilities and 5.2% show likely malicious intent, and can run as a CLI, MCP server, or install gate producing JSON/SARIF risk scores and DO_NOT_INSTALL/CAUTION/SAFE recommendations.

github.comrepoPublished and added 14 Jun 2026🛡️ Security

Tachi is a threat-modeling and AI-reasoning vulnerability detection harness that runs inside Claude Code, dispatching 14 specialized agents against an architecture description (Mermaid, C4, PlantUML, free-text, or ASCII) to find logic-level flaws like broken auth flows, missing privilege boundaries, prompt injection paths, and agent autonomy gaps that SAST cannot catch. It claims full 50/50 coverage across five OWASP frameworks (Web/API, LLM, Agentic, ML, Mobile Top 10s), classifies agentic AI findings against the CSA MAESTRO seven-layer taxonomy, and maps findings to MITRE ATT&CK/ATLAS, NIST AI RMF, and CWE, producing threats.md, SARIF, attack trees, quantitative risk scores, compensating-control analysis, and a PDF security report.

github.comrepoAdded 3 Sep 2026, published 23 May 2026🛡️ Security

A 258-item, 17-category security checklist targeting vulnerabilities that AI coding assistants (Claude, GPT, Gemini, Grok, Mistral, Copilot) commonly introduce, each item tagged with severity and a static/runtime/config detection method. Includes a companion prompt.md that turns the checklist into an automated scan runnable via Claude Code, Gemini CLI, or OpenAI Codex CLI against a local codebase, covering issues like prompt injection via user-supplied system prompts, missing auth middleware, hardcoded secrets, exposed API keys in frontend env vars, LLM-based auth decisions, unsandboxed agentic shell/filesystem access, and vector DB tenant isolation gaps.

github.comrepoAdded 3 Sep 2026, published 13 May 2026🛡️ Security

claude-code-security-review is an official Anthropic GitHub Action that uses Claude Code to perform diff-aware, semantic security analysis of pull requests, flagging issues like injection, broken auth, hardcoded secrets, insecure crypto, and RCE via deserialization, then posting findings as PR review comments. It includes false-positive filtering tuned to exclude low-impact findings such as DoS, rate limiting, and open redirects, and ships a matching security-review slash command for local use inside Claude Code; the authors explicitly warn it is not hardened against prompt injection and should only run on trusted, reviewed PRs.

github.comrepoPublished and added 10 May 2026🛡️ Security

MEDUSA is an AGPL-3.0 CLI security scanner (pip install medusa-security) built specifically for the AI/agentic era, shipping 40,000+ detection patterns across 79 scanner types for AI/ML apps, MCP servers, RAG pipelines, and traditional code, with zero external tool installation required. It detects AI supply-chain attacks like repo poisoning through weaponized editor configs (Cursor, Cline, Windsurf, Claude Code, Copilot, Kiro), MCP tool/schema poisoning, and prompt injection, and includes a `medusa secrets scan/purge` command that finds and redacts leaked API keys (21 issuer types) inside Claude Code, Cursor, Copilot, Gemini and shell REPL history files. It also carries a CVE database (200 entries) covering things like Log4Shell, XZ Utils, LangChain RCE, and the CVSS 10.0 React2Shell (CVE-2025-55182) RCE in React Server Components/Next.js.

github.comrepoPublished and added 10 May 2026🛡️ Security

DeepTeam is an open-source, locally-runnable red teaming framework for LLM applications, agents, and RAG pipelines, built on the DeepEval evaluation library. It simulates jailbreaking, prompt injection, and multi-turn adversarial attacks against 50+ vulnerability categories such as PII leakage, BOLA, BFLA, SQL injection, SSRF, and agentic goal theft, maps results to OWASP, NIST, MITRE, BeaverTails, and Aegis frameworks, and ships production guardrails to block the same issues at runtime.

github.comrepoPublished and added 7 May 2026🛡️ Security

A GitHub-hosted, structured learning roadmap for AI/ML security and prompt injection, covering prerequisites (web/API basics), LLM fundamentals (OWASP LLM Top 10 2025, MITRE ATLAS), attack classes (prompt injection, indirect prompt injection, jailbreaking, multi-turn attacks, MCP tool poisoning, RAG poisoning, model extraction, supply chain), and hands-on labs/tools (Garak, PyRIT, ART, CleverHans, ModelScan). It also catalogs named CVEs in AI coding assistants (CVE-2025-53773 GitHub Copilot RCE, CVE-2025-54135 Cursor IPI-to-RCE) and points to bug bounty platforms, checklists for pentesting LLM apps, and community resources (AI Village, Embrace the Red, r/llmsecurity).

github.comrepoPublished and added 7 May 2026🛡️ Security

mcpwn is a CLI security scanner for Model Context Protocol (MCP) servers, described as "nikto or nuclei, but for MCP servers." It runs checks (MCP-001 through MCP-013+) that statically analyze tool descriptions/schemas for hidden-instruction tool poisoning and, in an --aggressive mode, actively probe for prompt injection via tool output, SSRF against internal IPs/cloud metadata endpoints, command injection via shell metacharacters, cross-tool data exfiltration via canary tokens, and hardcoded secrets in resources. It supports stdio/SSE/HTTP transports, JSON/SARIF output for CI gating, and pairs with a companion runtime tool called mcp-firewall.

github.comrepoPublished and added 6 May 2026🛡️ Security

pentest-ai-agents is a GitHub repo providing 50+ Claude Code subagent markdown files that turn Claude into a routed offensive-security assistant, covering recon, web, Active Directory, cloud, mobile, wireless, social engineering, payload crafting (msfvenom/Donut), reverse engineering (Ghidra/JadX/Radare2), C2 tuning (Sliver/Mythic/Havoc/Cobalt Strike), container breakout, and LLM red-teaming (OWASP LLM Top 10, prompt injection, RAG poisoning, MCP abuse). It distinguishes Tier 1 advisory agents from Tier 2 execution-capable agents that enforce a declared-scope guard before running commands, and ships a SQLite findings database, hardened CI validator, and an offline Docker bundle.

github.comrepoPublished and added 6 May 2026🛡️ Security

AIMap is a Bishop Fox tool that discovers publicly exposed AI/ML endpoints (MCP servers, Ollama, vLLM/LiteLLM proxies, LangServe chains, Gradio apps, ComfyUI nodes) via 32+ curated Shodan queries, fingerprints them with Nuclei templates and live HTTP probes, and computes a 0-10 risk score based on auth, tool exposure, CORS, TLS, and system-prompt leakage. It also launches protocol-specific attack suites — MCP tool abuse, Ollama model extraction, prompt injection — with real-time streaming results and a Shodan-style query search interface plus a 3D globe visualization of findings.

github.comrepoPublished and added 6 May 2026🛡️ Security

A GitHub reference guide covering the OWASP Top 10 for LLM Applications 2025 and the OWASP Top 10 for Agentic Applications 2026 (ASI01–ASI10, released December 2025 at Black Hat Europe), with Python code samples for attacks (prompt injection, training-data extraction, RAG poisoning, agent goal hijacking, memory poisoning) and matching defenses (guardrails, memory integrity validation, tool-usage guards, zero-trust agent execution). It also curates red-teaming and guardrail tooling — garak, DeepTeam, Promptfoo, ARTKIT, Meta LlamaFirewall, LLM Guard, NeMo Guardrails, Amazon Bedrock Guardrails, Langfuse — plus an MCP security checklist and a security readiness checklist for LLM/agentic deployments.

github.comrepoPublished and added 5 May 2026🛡️ Security