Skip to content

Back to the catalog

github.comrepo

N4darae/shrt

shrt is a Go command-line tool for regression testing RPC backends. Agents, such as Claude Code through a SKILL.md skill, write per-domain contract overlays, and the planner composes RPC chains that probe boundaries, other roles, missing tokens and read-backs after writes. The tool runs and verifies chains against approved runs, and a gate command checks every new release. A person must approve any chain before it becomes a baseline.

TL;DR Go CLI that lets coding agents write contracts and generate, run and gate RPC regression chains, with human approval of baselines.
Document

What it claims

shrt's agent-facing docs (installed by shrt init into .shrt/docs/, routed from .claude/skills/shrt/SKILL.md) say that hand-written chains miss most regressions. The page's advice is to write contracts and let shrt contract plan compose the chains. The planner probes boundaries, the last item of a list, other roles, missing tokens, a read-back after every write, and repeats. The tool does the derivation. The agent supplies only test data and the assertions that define a correct response.

How it works

A contract is a per-domain YAML overlay under .shrt/contracts/. It holds required, failures, effects (what a write does to numbers, GRAMMAR.md §3) and fields.<f>.value, filled from the service's code. Field names and enum values must come from the descriptor (shrt catalog describe), never from memory.

Two checks differ in scope. run asks whether the assertions hold. verify diffs every response field against a person-approved run, including fields nobody asserted. An approved run is a "safe spot", and only a person can create one. Until one exists, shrt diff <name> compares the last two runs, which the page calls a comparison, not a verdict.

shrt gate sends every chain in .shrt/chains once, using verify where a safe spot exists and run otherwise, with a fresh -var tag. It retries an exit 3 once and holds shrt chain hollow (reads that passed but found nothing) to .shrt/hollow-baseline. If a token is refused early, the gate fetches a fresh one (at most 30s) and re-sends a read. A second refusal is reported as a finding that sessions end early. Each FAIL line names a suspect RPC and changed paths. Slices failing the same way fold into their parent's line.

flowchart TD
  C[chain in .shrt/chains] --> S{safe spot?}
  S -- yes --> V[verify: diff all fields]
  S -- no --> R[run: assertions only]
  V --> E{exit 3?}
  R --> E
  E -- yes, first time --> S
  E -- no --> L[PASS/FAIL line + suspect]

How to use it

The contract-first path, verbatim from the page:

export API_USER=... API_PASSWORD=...
shrt init
shrt contract init -all
shrt contract lint && shrt contract status -gaps
shrt contract plan -all -write
shrt run <chain>
shrt confirm -all -note "..."
shrt gate

On a fresh clone, run shrt init -agents=false -build=false, then shrt catalog build, which every catalog command needs, then shrt doctor. Extra roles need <ROLE>_USER/<ROLE>_PASSWORD exported at init, and the role must be named in the repo's README. A chain that is red on a correct backend is a real defect. shrt chain pin <chain> keeps it red in a slice while the rest stays green. Approval is shrt confirm <name> -approve -by <email>, and only after the user says yes to the proposal.

shrt init writes a CI wrapper to .shrt/ci-gate.sh, to be committed. The wrapper runs doctor -strict, contract lint, contract quality -gate -baseline .shrt/quality-baseline (only when overlays exist), chain lint -strict, then exec shrt gate. With CI set, a missing hollow baseline fails the gate. Outside CI, the first gate writes that baseline.

Exit codes: 0 is green or -h, 1 is a bad flag or setup, 2 is an unknown command, and 3 means no verdict, so re-run. Only run (not -dry-run), verify (not -run <id>), gate and chain slice -verify send traffic.

Limits

  • .shrt/docs/, the descriptor, .shrt/runs/, .shrt/tokens.json and pending safe spots are gitignored. A run id quoted in a document is therefore not checkable evidence.
  • The descriptor and the binary go stale quietly. Rebuild the descriptor after proto changes and the binary (go build -o shrt ./cmd/shrt) after changes to shrt itself.
  • shrt init guesses the auth: block and the login it picked should be checked. shrt run refuses to start without the env vars that auth.body reads.
  • A field that is ${vars.tag} alone drifts on every verify. Embed the tag in other text instead.
  • Latency fails the gate only with latency: {fail: true}.
  • shrt gate <chain>... on a subset skips the ratchet.
  • chain lint -strict fails only six assertion-quality warnings. Other warnings exit 0.
  • The page states no licence.

Organisations named here

  • Claude Code

Technologies named here

  • shrt
  • shrt contract plan
  • shrt gate
  • shrt verify
  • shrt confirm
  • shrt doctor
  • shrt catalog
  • Claude Code skills
  • Claude Code
  • Go

Techniques named here

  • Contract-driven chain planning
  • Run, verify and approve loop
  • Hollow-read detection