Skip to content

Back to the catalog

promptfoo
github.comrepo

promptfoo

promptfoo is an open-source CLI and library for evaluating and red-teaming LLM applications, letting developers run automated evals, compare model providers side-by-side, and scan for LLM-specific vulnerabilities like jailbreaks and prompt injection. It runs entirely locally, integrates into CI/CD for automated checks, and can generate security vulnerability reports and review pull requests for LLM-related security and compliance issues. The project was recently acquired by OpenAI but remains MIT-licensed and open source.

TL;DR Open-source CLI/library for LLM evaluation and red-teaming/vulnerability scanning, now part of OpenAI but still MIT-licensed.
Open original ↗
Document

What it is

promptfoo is a CLI and library for evaluating and red-teaming LLM apps. It is MIT licensed and, per the README, is now part of OpenAI. It compares providers side by side (OpenAI, Anthropic, Azure, Bedrock, Ollama and others), runs in CI/CD, and produces security vulnerability reports. It also has a code scanning feature that reviews pull requests for LLM-related security and compliance issues.

Preconditions / Affected

Most LLM providers need an API key set as an environment variable. The README works from an OpenAI key. The tool works with any LLM API or programming language.

What to do

Install with any one of these routes:

npm install -g promptfoo
brew install promptfoo
pip install promptfoo
npx promptfoo@latest

The last one runs any command without installing. Then scaffold the example project, set a key, run an eval and open the results:

promptfoo init --example getting-started
export OPENAI_API_KEY=sk-abc123
cd getting-started
promptfoo eval
promptfoo view

The sk-abc123 value is the README's placeholder. For red teaming and vulnerability scanning, the README points to its separate Red Teaming guide rather than giving commands on the page.

Caveats

The README says LLM evals run 100% locally and prompts never leave the machine. It also says most providers need an API key, so calls to a hosted model still go to that provider. The page does not say whether red teaming or PR code scanning run locally.

The "10M+ users" figure is the project's own claim and has no cited source. The page gives no version numbers, benchmarks or detection coverage, and no list of the vulnerability classes the scanner tests. The OpenAI acquisition is stated with a link to a company update, but the page does not describe any change to governance or roadmap.

Technologies named here

Techniques named here

  • Red teaming / vulnerability scanning of LLM apps

Related in AI Security