DeepTeam

What it is

DeepTeam is an Apache 2.0 Python red teaming framework from the founders of Confident AI, built on the DeepEval evaluation library. It runs locally and tests LLM apps, agents and RAG pipelines. It has a CLI driven by YAML configs and a Python API. The README pitches it as penetration testing for LLMs.

How it works

You supply only a model_callback (an async function that takes a string and returns your app's string output). You do not describe the system under test or provide a dataset. Attacks are generated dynamically from the vulnerabilities you select.

Each run pairs a vulnerability with one or more attacks. The attack enhances the baseline probe, and the callback's output is scored by a local LLM-as-a-Judge metric (for example BiasMetric). The score is binary, 0 or 1, with reasoning. The pass rate per vulnerability is the share of scores equal to 1. Results can be shown in dataframes and saved locally as JSON.

The README quantifies the catalog beyond what the summary lists:

• 50+ vulnerabilities in groups: Data Privacy, Responsible AI, Security, Safety, Business, Agentic and Custom. The agentic group includes Indirect Instruction, Tool Orchestration Abuse, Inter-Agent Communication Compromise and Autonomous Agent Drift. The security group includes Tool Metadata Poisoning, Cross-Context Retrieval and Debug Access.
• 20+ attacks. Single-turn examples are Base64, ROT13, Leetspeak, Multilingual, Context Flooding, Embedded Instruction JSON and Adversarial Poetry. Multi-turn attacks are Linear, Tree, Crescendo and Sequential Jailbreaking, plus Bad Likert Judge.
• Framework presets: OWASPTop10 (LLMs 2025), OWASP_ASI_2026 (Top 10 for Agents 2026), NIST, MITRE, Aegis and BeaverTails. Each maps its categories to vulnerabilities and attacks automatically.
• Custom vulnerabilities and attacks can be defined and plugged in.

What to do

Install and run a first test:

pip install -U deepteam

from deepteam import red_team
from deepteam.vulnerabilities import Bias
from deepteam.attacks.single_turn import PromptInjection

risk_assessment = red_team(
    model_callback=model_callback,
    vulnerabilities=[Bias(types=["race"])],
    attacks=[PromptInjection()]
)

To run against a standard instead of hand-picking:

from deepteam.frameworks import OWASPTop10
risk_assessment = red_team(model_callback=model_callback, framework=OWASPTop10())

Runtime guardrails are a separate API with input and output guards:

from deepteam import Guardrails
from deepteam.guardrails import PromptInjectionGuard, ToxicityGuard, PrivacyGuard

guardrails = Guardrails(
    input_guards=[PromptInjectionGuard(), PrivacyGuard()],
    output_guards=[ToxicityGuard()]
)
guardrails.guard_input("...").breached
guardrails.guard_output(input="...", output="...").breached

The seven guards are ToxicityGuard, PromptInjectionGuard, PrivacyGuard, IllegalGuard, HallucinationGuard, TopicalGuard and CybersecurityGuard.

Caveats

• Judge and attack generation need an LLM. The README tells you to set OPENAI_API_KEY, or use any custom model DeepEval supports. "Local" means the framework and metrics run on your machine, not that no model calls are made.
• Scoring is LLM-as-a-Judge, so results depend on the judge model chosen. The page gives no accuracy or benchmark figures.
• The README's example callback returns a canned refusal. You must replace it with your real application to get meaningful results.
• Result tracking, production monitoring, report sharing and the MCP server for Cursor and Claude Code belong to the paid-platform Confident AI. They are not part of the open-source package.
• The page does not describe guardrail latency or accuracy, even though it calls them "fast".