yolorouter/yolorouter

What it claims

Yolorouter is an Apache 2.0, single-binary Go gateway with the web console embedded. It needs no Node runtime and no separate frontend. It uses SQLite by default, with PostgreSQL as an option. Clients change only their base URL and API key. The public model name you configure, e.g. smart, is kept in responses while Yolorouter swaps in each provider's real model id. Beyond the four chat protocols, it also serves the OpenAI Images, Videos, and Speech APIs.

How it works

Each public model maps to an ordered list of provider candidates. Each candidate holds a pool of upstream keys.

• Failover mode (the default): the first candidate takes all traffic.
• Balanced mode: each caller API key is bound to the provider holding the fewest keys, then sticks to it so upstream prompt caches stay warm. When a provider trips the circuit breaker, keys re-bind as they next call. The recovered provider then attracts new assignments, with no separate rebalancer.

flowchart TD
  R[Request, public model] --> M{Scheduling mode}
  M -->|failover| P[Head candidate]
  M -->|balanced| B[Key's bound provider]
  P --> K[Round-robin upstream key]
  B --> K
  K -->|429| BN[Bench for Retry-After, next key]
  K -->|401 / quota| OUT[Remove until retest passes]
  K -->|fails before first byte| NX[Next candidate]
  K -->|first byte sent| L[Provider locked, never stitched]

Other mechanics the page describes:

• Timeouts: seven independently configurable phases rather than one wall clock, so long-thinking models aren't killed.
• Vision fallback: images sent to text-only models are described by a chosen vision model and forwarded as text. If none is configured, the image becomes a placeholder.
• Input compression: recognises go test output, git diffs, grep results, and logs. It keeps failures, stack traces, and each distinct match. It never touches the active edit region at the conversation tail, and only replaces a block if the result is shorter.
• System-prompt injection: byte-identical across requests, so it stays cache-friendly.
• Cost accounting: cache-read and cache-write tokens are priced separately.

Evidence

The system-prompt savings rest on a paired on/off benchmark with 150 raw measurement pairs, published in docs/concise-output-benchmark.md. The page gives no headline numbers.

How to use it

docker run -d --name yolorouter --restart unless-stopped \
  -p 8080:8080 -v "$PWD/yolorouter:/yolorouter" \
  ghcr.io/yolorouter/yolorouter:latest

Images are published for amd64 and arm64. All state lives in the mounted folder: configs/config.yaml, which includes the key that encrypts your upstream keys, plus the SQLite database. Use an absolute -v path; otherwise rerunning from another directory yields an empty setup screen. Pin a tag (e.g. :v0.1.6) to avoid surprise migrations.

Service install with the built-in self-updater (a China mirror is available at gh.yolorouter.com):

curl -fsSL https://get.yolorouter.com/install.sh | bash

Claude Code via the gateway:

export ANTHROPIC_BASE_URL=http://localhost:8080
export ANTHROPIC_AUTH_TOKEN=sk-yr-your-key
claude

Building from source requires Go 1.25.7+ and Node.js 22.12+ (make build-embed). The docs include setup guides for 19 agent tools.

Limits

Protocol translation

• On the Responses ingress, input_image entries are dropped when translating to a different egress protocol; only text is forwarded.

Balanced mode

• Bindings live in process memory and are lost on restart.
• In multi-instance deployments, each instance computes its own spread.
• The binding table is capped at 4096 (key, model) pairs, with LRU eviction beyond that.
• Old instances in a rolling upgrade treat every model as failover.

Upgrades

• Migrations run automatically on upgrade. Rolling back requires restoring a pre-upgrade backup, not just starting an older image.

Media

• Images on DashScope/Kling hosts return URLs only; b64_json is refused, and masks are refused on DashScope.
• Image streaming works only for gpt-image-*.
• Returned media URLs follow upstream expiry; the gateway never rehosts them.
• Video jobs cannot be cancelled and are never resubmitted to another candidate.
• MiniMax-H3-Max accepts only 5–15 s clips. MiniMax pending jobs expire unbilled after 7 days, and video bills the pay-as-you-go balance only.

Speech

• Speech never fails over, because voices don't transfer between vendors.
• instructions and stream_format are refused.
• The same text may be metered differently per provider.