Skip to content

Back to the catalog

rewanthtammana/Who-Let-The-Agents-Act
github.comrepo

rewanthtammana/Who-Let-The-Agents-Act

who-let-the-agents-act is a self-hostable lab suite (Docker/Python, Groq-backed LLM agent) with nine isolated scenarios — overpowered data tools, IDOR-style cross-account access, unsafe agent handoffs, refund-limit bypass, indirect prompt injection via poisoned invoices, multi-tenant RAG leakage, secret leakage through debugging, fail-open approval outages, and a multi-hop confused-deputy agent chain. Each scenario runs in multiple modes (vulnerable vs. application-enforced) against the same prompt, recording model plan, policy decision, tool calls, and DB access as JSON so a reader can directly compare what changes when authorization is enforced outside the system prompt.

TL;DR Nine runnable labs showing how AI agents with excessive tool/data access become IDOR, prompt-injection, and confused-deputy vulnerabilities — with full decision/trace comparison between vulnerable and enforced modes.
Open original ↗
Document

What it is

who-let-the-agents-act is a self-hostable set of nine agent-security labs (Apache-2.0, Python/Docker, Groq-backed LLM). The summary lists the scenarios; what it omits is the enforcement architecture the labs are built to demonstrate and the exact commands to stand it up and validate it.

How it works

The design point the labs make concrete: the model is never trusted to authorize. It only proposes an intent, identifier, field, or action; application code validates that proposal and makes the final decision before any database or side-effecting tool is reached. Each scenario runs the same request across three modes (the summary says only "vulnerable vs. application-enforced"), and each run writes its model plan, policy decisions, tool calls, DB access, and outcome as JSON under the scenario's _runs/ directory. The stated thesis: a system prompt is not an authorization boundary, and identity, tenant, field, business-rule, approval, provenance, and DLP checks belong in application code.

flowchart LR
    A[User request] --> M[Model: propose intent/id/field/action]
    M --> P{Application policy}
    P -- allow --> D[(DB / side-effecting tool)]
    P -- deny --> X[Blocked + trace]

What to do

Run locally:

python3 -m pip install -r requirements.txt
cp .env.example .env
# Set GROQ_API_KEYS in .env; GROQ_API_KEY also works for one key.
python3 run.py

Or via Docker (docker compose up --build -d); either serves the UI at http://127.0.0.1:8000, where you pick scenario, mode, and prompt. Backend/scenario edits hot-reload; rebuild the image only when dependencies change.

Static checks and tests:

python3 scripts/validate_scenarios.py
python3 -m py_compile app.py run.py core/*.py scenarios/*/*.py scenarios/*/modes/*.py scenarios/*/tools/*.py
python3 -m unittest discover -s tests -v
node --check static/app.js && node --check static/blog.js && node --check static/theme.js && node --check static/github-callout.js

With Groq configured, verify one scenario or the full live matrix:

python3 scripts/verify_matrix.py --scenario overpowered-data-tool
python3 scripts/verify_matrix.py

Caveats

Vulnerable modes are intentional. All data is synthetic; the project explicitly warns against connecting vulnerable modes to production data, credentials, accounts, or payment systems. Deployment and configuration specifics live in TECHNICALS.md, not the README.

Technologies named here

Techniques named here

  • Overpowered Data Tool
  • Cross-Account Access
  • Unsafe Agent Handoff
  • Refund Limit Bypass
  • Poisoned Invoice Instructions
  • Multi-tenant RAG Leakage
  • Secret Leakage Through Debugging
  • Approval Service Outage
  • Confused Deputy Agent Chain
  • IDOR

Related in AI Security