Skip to content

Back to the catalog

promptfoo.devtool

PromptFoo LM Security DB

PromptFoo's LM Security DB is a searchable database of primary-source AI security research findings, indexing 985 findings across 1123 evaluated models pulled from papers on arXiv and elsewhere. Sampled entries cover memory-deletion leakage persisting in agent summaries/KV cache, embedding-inversion leakage in retrieval systems (SHAQ), vision-language model over-refusal (KoNA), document-image prompt injection against VLM agents (Repeat-After-Me), PII leakage in document images (LeakageBench), context-injection promoting untrusted repo/tool content into agent instructions, and cross-tenant GPU isolation violations in LLM serving frameworks like vLLM and DistServe (GIFT). It is positioned as a research-tracking reference for AI red-teamers and defenders rather than an original piece of research itself.

TL;DR Searchable database of 985+ primary-source AI/LLM security research findings across 1123 evaluated models, spanning agent memory poisoning, embedding inversion, VLM jailbreaks, and multi-tenant LLM-serving isolation bugs.
Open original ↗

Technologies named here

Techniques named here

  • Boilerplate refusal statements causing unnecessary refusals
  • Retrieval embedding leakage
  • Document-image PII localization/detection gaps
  • Context assembly privilege promotion
  • Shared GPU state isolation violation
  • Persistent memory-store response manipulation
  • Third-party skill resource inflation

Related in AI Security