ShadowSafin/Synapass

What it claims

Synapass is an Apache 2.0 inference gateway exposing POST /v1/chat/completions on port 8080, with an operator dashboard on port 3000. Any OpenAI client works by changing its base URL. Each response carries an extra synapass block with the request ID, the provider that actually answered, the requested and routed model, whether fallback was used, latency, and estimated cost in USD. OpenAI SDKs ignore this block. Routing reasons are read back from GET /admin/v1/requests/{id}/explain.

How it works

Each request passes nine stages in order: authenticate (API key to tenant), resolve the most specific policy, enforce budgets and rate limits, check the cache (exact, then prefix, then semantic), classify the task (rules first), shape the prompt (normalize, trim), route (filter, order, pick), execute with retry and failover, and run a bounded tools loop when allowed. Telemetry is written asynchronously.

flowchart TD
  A[Auth + policy] --> B{Budget or policy denial?}
  B -- yes --> X[403/429, no provider call]
  B -- no --> C{Cache hit?}
  C -- yes --> R[Response + synapass block]
  C -- no --> D[Classify, shape, route]
  D --> E[Provider call]
  E -- failure --> F[Retry / fail over] --> E
  E --> R

The stores are split by role: PostgreSQL is the system of record; Redis holds limits, cache and credentials; NATS carries usage, eval jobs and audit; workers write to ClickHouse, which feeds OTel, Prometheus, Loki and Grafana. Code entry points are internal/api/chat.go (request path) and internal/routing (decision logic). The test suite covers Go and Python.

How to use it

One-line install, which clones into ~/synapass (override with SYNAPASS_DIR), generates .env with an admin key, starts every Compose service and waits for readiness:

curl -fsSL https://raw.githubusercontent.com/shadowsafin/synapass/main/scripts/install.sh | bash

On native Windows PowerShell:

irm https://raw.githubusercontent.com/shadowsafin/synapass/main/scripts/install.ps1 | iex

From a clone, run ./scripts/deploy.sh with stop|restart|logs|status|reset. The reset option deletes volumes after a --yes confirmation. Re-running the deploy script updates the checkout and leaves .env and data untouched.

Add OPENAI_API_KEY or ANTHROPIC_API_KEY to .env afterwards. These are the only two provider keys the page names. Without a key, the stack boots and providers report as unconfigured. If port 8080 is taken, set GATEWAY_PORT and NEXT_PUBLIC_SYNAPASS_API_URL to change the host port. scripts/smoke.sh checks the stack end to end without a provider key. Tenant keys are minted through POST /admin/v1/keys with a tenant_id and scopes such as inference. The plaintext key is returned once.

Besides Docker, the page lists other ways to run it: synapass native install and synapass native up, systemd units in deploy/systemd/, and a Windows desktop app (SynapassSetup.exe, Synapass.exe).

Limits

• The full stack is heavy: gateway, dashboard, Postgres, Redis, ClickHouse, NATS, workers and a Grafana/Prometheus/Loki stack all start together.
• Only /v1/chat/completions is shown. Other OpenAI endpoints are not mentioned.
• Only control-plane changes are audited. Inference traffic is not; usage records serve that purpose instead.
• The gateway refuses to start on unsafe production configuration: a CORS wildcard, a missing admin key, or tracing enabled with no OTLP endpoint.
• Truncated answers are flagged with "truncated": true and a reason such as max_tokens.
• The page gives no benchmarks, latency overhead or throughput figures. The single example response shows 2851 ms latency end to end.