Awesome LLM Security

What it is

The corca-ai/awesome-llm-security list is a link index of LLM security papers (dated May 2022 to March 2025), tools, a CVE, competitions and blogs. Papers carry tags such as multi-modal, transfer, gene-algorithm, cipher, ICL, token-prob and self-filtered, plus venue and links to paper, repo, site or dataset. The page adds no analysis of its own. A note says PDFs are shared via Moonlight, which pairs each paper with a summary.

What to do

The page's practical value is the set of items you can run or measure against.

Benchmarks and evaluation harnesses (each has a repo link unless noted):

• JailbreakBench (March 2024): open robustness benchmark for jailbreaking.
• AgentDojo (NeurIPS 24): dynamic environment for attacks and defenses on LLM agents.
• Formalizing and Benchmarking Prompt Injection Attacks and Defenses (USENIX Security 24).
• AgentHarm (October 2024): harmfulness of LLM agents. The page links only the paper.
• Do-Not-Answer: dataset for evaluating safeguards.
• Rethinking How to Evaluate Language Model Jailbreak (April 2024).
• Open-Prompt-Injection and jailbreak-evaluation (a Python package).

Scanners and guards:

• Garak: LLM vulnerability scanner.
• PurpleLlama: tools to assess and improve LLM security.
• LLMFuzzer: fuzzing framework.
• Prompt Fuzzer: hardens GenAI applications.
• Agentic Radar: CLI security scanner for agentic workflows.
• WhistleBlower: infers an agent's system prompt from its outputs.
• Rebuff, Vigil and LLM Guard: prompt injection detection and filtering.

Defenses with code linked: Baseline Defenses (perplexity, paraphrase-style baselines), LLM Self Defense, AutoDefense (multi-agent), Information Bottleneck, PARDEN (repetition), and Circuit Breakers (NeurIPS 24). Detecting Language Model Attacks with Perplexity (August 2023) is listed without a repo.

Other entries:

• LangChain code injection, CVE-2023-29374, is listed as a link only, with no mechanism or fix.
• The Mozilla 0din GenAI bug bounty covers prompt injection, training data poisoning and DoS.
• Gandalf (prompt injection wargame) and the DEFCON31 Quals LLM challenges are listed for practice.

Caveats

• Entries are one-line descriptions. The page gives no results, no comparison between tools, and no guidance on which defense holds against which attack.
• Coverage of papers appears to end around March 2025, with one agent benchmark from October 2024. Newer work is not captured.
• The tool list mixes detectors, scanners and one tool-calling protocol (UTCP). Maintenance status of each is not stated.
• Some papers are listed without a repo or a venue, so reproducibility has to be checked per item.