Skip to content

Back to the catalog

requie/LLMSecurityGuide
github.comrepo

requie/LLMSecurityGuide

A GitHub reference guide covering the OWASP Top 10 for LLM Applications 2025 and the OWASP Top 10 for Agentic Applications 2026 (ASI01–ASI10, released December 2025 at Black Hat Europe), with Python code samples for attacks (prompt injection, training-data extraction, RAG poisoning, agent goal hijacking, memory poisoning) and matching defenses (guardrails, memory integrity validation, tool-usage guards, zero-trust agent execution). It also curates red-teaming and guardrail tooling — garak, DeepTeam, Promptfoo, ARTKIT, Meta LlamaFirewall, LLM Guard, NeMo Guardrails, Amazon Bedrock Guardrails, Langfuse — plus an MCP security checklist and a security readiness checklist for LLM/agentic deployments.

TL;DR A comprehensive OWASP-aligned LLM/agentic AI security guide with code examples for prompt injection, jailbreaks, RAG/agent attacks, memory poisoning, and matching defenses/tooling.
Document

What it is

requie/LLMSecurityGuide is a single-README guide, last updated February 2026, that maps the OWASP Top 10 for LLM Applications 2025 (released 18 November 2024) and the OWASP Top 10 for Agentic Applications 2026 (ASI01 to ASI10, released 10 December 2025) to attack snippets, defense snippets and tool lists. The summary covers that scope. This document covers the incident write-ups and the mappings the summary leaves out. The repository is MIT licensed and community maintained.

Incident write-ups with figures

  • EchoLeak, CVE-2025-32711: a zero-click indirect prompt injection in Microsoft 365 Copilot. Character substitutions bypassed safety filters and made the assistant send business data to an external URL. The guide's lesson is to require human approval for any action that sends data outside.
  • DeepSeek R1:
  • CrowdStrike measured the code vulnerability rate rising from 19% to 27.2% when prompts touched politically sensitive topics.
  • Qualys ranked it 17th of 19 models, with a 77% attack success rate against 27% for OpenAI o1-preview, and it failed 58% of jailbreak tests.
  • Enkrypt AI found it 11x more likely to produce harmful output than o1 and 4x more likely to produce insecure code.
  • Trend Micro showed exposed <think> tags enabling guardrail bypass.
  • An unauthenticated ClickHouse database was left publicly accessible.
  • Malicious MCP server on npm, September 2025: described as the first, and tied to ASI04 (agentic supply chain).
  • Others: the Air Canada chatbot fare dispute, Samsung source code pasted into ChatGPT, and an Anthropic disclosure of agents used in espionage campaigns. The page gives root causes but no dates or figures for these.

ASI mapping used throughout

The guide tags each defense with an ASI id. It names these ids:

  • ASI01: goal hijack
  • ASI02: tool misuse
  • ASI03: NHI and privilege abuse
  • ASI04: supply chain
  • ASI06: memory and context poisoning
  • ASI10: rogue agents

Its full "Agentic Top 10 Security" checklist also lists controls for ASI05, ASI07, ASI08 and ASI09:

  • ASI05: code execution sandboxing
  • ASI07: inter-agent communication signing
  • ASI08: cascading failure circuit breakers
  • ASI09: human oversight for high-risk actions

The guide also states OWASP's "least agency" principle and points to the AIVSS calculator at aivss.owasp.org.

What to do

The MCP checklist is the most directly usable part:

  • Verify MCP server authenticity before connecting.
  • Keep an allowlist of permitted servers.
  • Audit a server's capabilities before enabling it.
  • Monitor MCP communications.
  • Rate limit MCP calls.
  • Validate server outputs before use.

For ASI03 it suggests task-scoped, time-limited agent credentials, with a 15-minute TTL in its example. The page shows these commands for scanning:

pip install garak
garak --model openai --model-name gpt-4 --probes all
garak --model huggingface --model-name microsoft/DialoGPT-medium --probes promptinject
pip install deepteam
deepteam scan --model openai --attacks all --vulnerabilities owasp-top-10

Caveats

  • Most code samples are skeletons that call undefined components, such as GoalConsistencyValidator and ToolOutputValidator. They show structure and cannot be run as written.
  • The memory-poisoning defense matches a handful of regexes (for example ignore.*previous). That is a starting point, not a control.
  • The page does not link primary sources for the incident figures inline. Check CrowdStrike, Qualys and the CVE record before citing them.
  • The deepteam scan command is shown as the page gives it, without a source for the syntax. Check it against the DeepTeam documentation before relying on it.
  • The page's own disclaimer says it offers no warranty of completeness or accuracy.

Organisations named here

  • Air Canada
  • DAN

Technologies named here

  • ChatGPT
  • garak
  • DeepTeam
  • Claude
  • promptfoo
  • Gemini
  • ARTKIT
  • GitHub Copilot
  • CodeT5
  • Meta LlamaFirewall
  • Amazon Bedrock Guardrails
  • DeepSeek R1
  • Llama 4
  • Langfuse
  • WildGuard
  • Amazon Bedrock
  • Air Canada customer service chatbot
  • AEGIS 2.0
  • BingoGuard
  • PolyGuard
  • OmniGuard
  • LLM Guard
  • NeMo Guardrails
  • AIVSS
  • Model Context Protocol
  • LangChain
  • LLaMa
  • Hugging Face
  • OWASP Top 10 for LLM Applications
  • NIST AI RMF

Techniques named here

  • Prompt Injection
  • Adversarial Suffixes
  • Multilingual Injection
  • Encoding Attacks
  • Indirect prompt injection
  • Jailbreaking
  • Sensitive Information Disclosure / Data Extraction
  • Model Inversion / Training Data Extraction
  • RAG/Vector Database Attacks
  • Chain-of-Thought Jailbreak
  • Multimodal Jailbreak
  • Multimodal Injection
  • Payload splitting
  • Agentic Manipulation
  • Goal Hijacking
  • ASI02
  • ASI03
  • ASI06
  • ASI10
  • Excessive Agency / Excessive Permissions
  • Action Hijacking / Chain-of-Thought Attacks on agents