Skip to content

Back to the catalog

autoguardrails
github.comrepo

autoguardrails

autoguardrails is an open-source alignment research scaffold from Santander AI Lab, inspired by Karpathy's autoresearch pattern, that searches over a single mutable policy.md file to minimize attack success rate (ASR) against a fixed jailbreak/obfuscation eval suite while enforcing a benign-pass floor so the guardrail can't just refuse everything. It reports asr_unguarded, asr_with_policy, and policy_delta to isolate the guardrail's contribution from the target model's native safety, and supports both an offline deterministic stub and real OpenAI-compatible target/judge models.

TL;DR Open-source harness that iteratively evolves a guardrail policy.md file to reduce jailbreak attack success rate against a frozen eval suite, with a benign-pass floor.
Open original ↗
Document

What it is

autoguardrails (Santander AI Lab) is an alignment research scaffold modeled on Karpathy's autoresearch, but searching over a single policy.md guardrail prompt instead of train.py. The summary covers the metrics; what follows is the operating procedure, the acceptance rule, and the scope limits the summary omits.

How it works

You edit exactly one file between runs (policy.md); everything else — eval_suite.jsonl, judge_prompt.md, the autoguardrails/ harness — is fixed. The concrete acceptance rule: a candidate is kept only if ASR improves and benign pass does not fall by more than 2 percentage points. Rejected candidates trigger an automatic restore of policy.md to the last accepted version. Each evaluation pass runs under a fixed wall-clock budget, currently 5 minutes. results.tsv is an append-only log.

What to do

Run from the repo root:

python -m autoguardrails baseline --reset --repeat 2 --notes "initial baseline"
# edit only policy.md
python -m autoguardrails candidate --repeat 2 --notes "cover jailbreak and obfuscation"
python -m autoguardrails status
cat results.tsv

Or via the wrapper: sh run_autoguardrails.sh baseline "initial baseline" 2.

For real (non-stub) runs, point target and judge at OpenAI-compatible endpoints:

export AUTOGUARDRAILS_TARGET_PROVIDER=openai_compatible
export AUTOGUARDRAILS_TARGET_MODEL=gpt-4.1-mini
export AUTOGUARDRAILS_TARGET_API_BASE=https://your-endpoint.example/v1
export AUTOGUARDRAILS_TARGET_API_KEY=your-target-key
export AUTOGUARDRAILS_JUDGE_PROVIDER=openai_compatible
export AUTOGUARDRAILS_JUDGE_MODEL=gpt-4.1-mini
export AUTOGUARDRAILS_JUDGE_API_BASE=https://your-endpoint.example/v1
export AUTOGUARDRAILS_JUDGE_API_KEY=your-judge-key

Change one policy family at a time. A useful first candidate: add explicit handling for jailbreak phrasing ("ignore previous instructions", "roleplay", "developer mode") and obfuscation (translation, base64, rot13, JSON-only, schema conversion).

Caveats

  • Freeze the judge (prompt and model) across a run series; switching mid-experiment invalidates comparisons.
  • The eval suite is fixed by design — if you edit it, start a new experiment lineage rather than comparing against old results.
  • Intentionally single-turn and narrow: no tools, file access, or multi-step agent actions modeled.
  • The bundled stub is for harness verification only — not a realistic safety model; the "improvement curve" it produces is illustrative.
  • Requirements: Python 3.10+, no third-party runtime deps (stdlib only, offline by default). Windows users need Git Bash or another POSIX shell for the wrapper.
flowchart TD
    A[Edit policy.md] --> B[Run candidate]
    B --> C{ASR improves AND<br/>benign drop ≤ 2pp?}
    C -->|yes| D[Keep candidate]
    C -->|no| E[Auto-restore last accepted policy.md]
    D --> A
    E --> A

Technologies named here

Techniques named here

  • Jailbreak phrasing attacks
  • Obfuscation requests

Related in AI Security