respanai/respan-gateway

What it claims

respan-gateway is a single Rust binary, dual-licensed MIT or Apache-2.0, that exposes one endpoint for OpenAI, Anthropic, Gemini and 15 other vendors. Beyond chat, it covers realtime voice (OpenAI Realtime ⇄ Gemini Live), images, speech, transcription, embeddings and video jobs. Unrecognised requests are relayed untouched. Built-in vendors are OpenAI, Anthropic, Gemini, Respan, TypeSafe, Groq, DeepSeek, OpenRouter, xAI, Mistral, Together, Fireworks, Ollama, Azure, vLLM, Cloudflare, Cerebras and Perplexity, plus any compatible endpoint. Twelve "tool vendors" (Apify, Browserbase, Cloudinary, Context7, Daytona, E2B, ElevenLabs, Linkup, Reducto, Tavily, TinyFish, Zep) turn an HTTP API into MCP tools with one key. Traces are sent to Respan.

How it works

No config file is required. Every vendor whose API key is in the environment becomes a provider. The model name picks the provider, whatever API format the client speaks. A provider/model name such as groq/openai/gpt-oss-20b selects a provider explicitly. Named routes map an alias to an ordered list of targets. For clients that know nothing of MCP, the gateway can run MCP tools inside chat requests through a tool loop. It also serves a merged /mcp endpoint. Every feature is a plugin on one public Rust API. The examples/ directory has one-file plugins for guardrails, PII masking, billing and an LLM judge that picks the model.

How to use it

No release is tagged yet, so the page says to build from source:

cd apps/ui && pnpm install && pnpm build
cargo build --release -p respan-gateway

After the first tag, it will also be available through install.sh/install.ps1, cargo install respan-gateway, pip install respan-gateway and GitHub releases. Run it with keys in the environment:

export OPENAI_API_KEY=sk-... ANTHROPIC_API_KEY=sk-ant-... GEMINI_API_KEY=...
respan-gateway

It listens on localhost:8080. Point an SDK's base URL at http://localhost:8080/v1 for OpenAI or http://localhost:8080 for Anthropic. Media uses OpenAI paths: /v1/images/generations, /v1/audio/transcriptions, /v1/audio/speech, /v1/embeddings, /v1/videos. GET /v1/models lists all providers' models, and the UI is at /ui/. respan-gateway init writes a commented gateway.toml:

client_keys = [{ name = "app", key = "env:APP_KEY" }]
[providers.local]                       # Any OpenAI-compatible endpoint
formats = ["openai-chat"]
base_url = "http://localhost:8000/v1"
[routes.smart]                          # Clients ask for "model": "smart"
to = ["anthropic/claude-haiku-5-5", "openai/gpt-6-luna"]

To embed it as a library, the crate is respan_gateway: App::new(config)?.router() returns an axum::Router.

Limits

No release was tagged at the time of the page. The one-line install scripts and package installs are documented, but the page directs users to build from source until the first tag. Without client_keys configured, the gateway answers only loopback and refuses to start on a public address. Vendors are assigned "tiers," which are documented on a separate Vendors page and not explained here. Translation fidelity for tools, images and reasoning is described only in per-format READMEs. The page gives no benchmarks or latency figures for the gateway itself.