N4darae/shrt
shrt is a Go command-line tool for regression testing RPC backends. Agents, such as Claude Code through a SKILL.md skill, write per-domain contract overlays, and the planner composes RPC chains that probe boundaries, other roles, missing tokens and read-backs after writes. The tool runs and verifies chains against approved runs, and a gate command checks every new release. A person must approve any chain before it becomes a baseline.
What it claims
shrt's agent-facing docs (installed by shrt init into .shrt/docs/, routed from .claude/skills/shrt/SKILL.md) say that hand-written chains miss most regressions. The page's advice is to write contracts and let shrt contract plan compose the chains. The planner probes boundaries, the last item of a list, other roles, missing tokens, a read-back after every write, and repeats. The tool does the derivation. The agent supplies only test data and the assertions that define a correct response.
How it works
A contract is a per-domain YAML overlay under .shrt/contracts/. It holds required, failures, effects (what a write does to numbers, GRAMMAR.md §3) and fields.<f>.value, filled from the service's code. Field names and enum values must come from the descriptor (shrt catalog describe), never from memory.
Two checks differ in scope. run asks whether the assertions hold. verify diffs every response field against a person-approved run, including fields nobody asserted. An approved run is a "safe spot", and only a person can create one. Until one exists, shrt diff <name> compares the last two runs, which the page calls a comparison, not a verdict.
shrt gate sends every chain in .shrt/chains once, using verify where a safe spot exists and run otherwise, with a fresh -var tag. It retries an exit 3 once and holds shrt chain hollow (reads that passed but found nothing) to .shrt/hollow-baseline. If a token is refused early, the gate fetches a fresh one (at most 30s) and re-sends a read. A second refusal is reported as a finding that sessions end early. Each FAIL line names a suspect RPC and changed paths. Slices failing the same way fold into their parent's line.
flowchart TD
C[chain in .shrt/chains] --> S{safe spot?}
S -- yes --> V[verify: diff all fields]
S -- no --> R[run: assertions only]
V --> E{exit 3?}
R --> E
E -- yes, first time --> S
E -- no --> L[PASS/FAIL line + suspect]How to use it
The contract-first path, verbatim from the page:
export API_USER=... API_PASSWORD=...
shrt init
shrt contract init -all
shrt contract lint && shrt contract status -gaps
shrt contract plan -all -write
shrt run <chain>
shrt confirm -all -note "..."
shrt gateOn a fresh clone, run shrt init -agents=false -build=false, then shrt catalog build, which every catalog command needs, then shrt doctor. Extra roles need <ROLE>_USER/<ROLE>_PASSWORD exported at init, and the role must be named in the repo's README. A chain that is red on a correct backend is a real defect. shrt chain pin <chain> keeps it red in a slice while the rest stays green. Approval is shrt confirm <name> -approve -by <email>, and only after the user says yes to the proposal.
shrt init writes a CI wrapper to .shrt/ci-gate.sh, to be committed. The wrapper runs doctor -strict, contract lint, contract quality -gate -baseline .shrt/quality-baseline (only when overlays exist), chain lint -strict, then exec shrt gate. With CI set, a missing hollow baseline fails the gate. Outside CI, the first gate writes that baseline.
Exit codes: 0 is green or -h, 1 is a bad flag or setup, 2 is an unknown command, and 3 means no verdict, so re-run. Only run (not -dry-run), verify (not -run <id>), gate and chain slice -verify send traffic.
Limits
.shrt/docs/, the descriptor,.shrt/runs/,.shrt/tokens.jsonand pending safe spots are gitignored. A run id quoted in a document is therefore not checkable evidence.- The descriptor and the binary go stale quietly. Rebuild the descriptor after proto changes and the binary (
go build -o shrt ./cmd/shrt) after changes to shrt itself. shrt initguesses theauth:block and the login it picked should be checked.shrt runrefuses to start without the env vars thatauth.bodyreads.- A field that is
${vars.tag}alone drifts on every verify. Embed the tag in other text instead. - Latency fails the gate only with
latency: {fail: true}. shrt gate <chain>...on a subset skips the ratchet.chain lint -strictfails only six assertion-quality warnings. Other warnings exit 0.- The page states no licence.
Organisations named here
- Claude Code
Technologies named here
- shrt
- shrt contract plan
- shrt gate
- shrt verify
- shrt confirm
- shrt doctor
- shrt catalog
- Claude Code skills
- Claude Code
- Go
Techniques named here
- Contract-driven chain planning
- Run, verify and approve loop
- Hollow-read detection
Related in coding-agents
