Folkbench/folkbench

What it claims

Switch, the desktop client half of Folkbench, is licensed Apache-2.0. It writes only the connection parameters (Base URL, API Key) into Claude Code or Codex config files and leaves the rest of each tool's custom settings in place. The page says all keys and configs stay on local disk in the user's home directory and nothing is uploaded. The per-session usage estimate parses only token counts locally. It does not read or upload code, prompts or conversation context. Cost estimates use "public reference prices", not the relay's actual billing. The client also shows the benchmark site's live availability and latency for each relay, so you can check a line before switching to it.

How it works

The page describes four steps:

1. Add a provider: name, Base URL and API Key.
2. Pick the target tool (Claude Code or Codex) in the sidebar.
3. Enable a line. The config is written and takes effect immediately, with a diff preview shown before the write.
4. Switch back, or clear the entry, to return to official or another provider.

The Codex path branches on the target:

flowchart LR
  A[Select line for Codex] --> B{Target?}
  B -->|Third-party relay| C[Stash official ChatGPT OAuth]
  C --> D[Write Base URL + Key]
  B -->|Official| E[Restore stashed OAuth]
  D --> F[Codex config]
  E --> F

How to use it

• Download: installers are on GitHub Releases, one per platform, and each release ships SHA256 checksums.
• Windows warning: the Windows builds are not yet code-signed ("in progress"). The page tells users to click through the OS security prompt with "仍要打开" ("open anyway").
• Security reports: go privately via SECURITY.md.
• Your own bug reports: the page warns you to redact real API keys from issues and log screenshots.

Limits

• Unsigned binaries. The Windows binaries are unsigned. Verify the SHA256 checksum before you click through the warning. Consider building from source if this tool will hold API keys.
• No published method. The README gives no probe methodology: no node count or locations, test prompts, request rate, or definition of "availability". It also contains no actual benchmark numbers; those live only on the external site. Treat the rankings as unverified until you have checked the method there.
• Promotional relay profiles. The relay section is advertising copy, with no stated relationship between Folkbench and the relays it profiles. It quotes:
• Modelflare: daily cache-hit guarantees of 65/75/85% by tier; a GPT discount tier unlocked by a US$20 top-up at about ¥0.51 input / ¥3.01 output per million tokens; a Claude tier unlocked by US$50.
• ApiPorter: a ¥5 minimum top-up.
• Several relays are self-described as new, with little long-term data.

None of these claims is tied to Folkbench measurements in the README. A benchmark operator that also promotes the services it ranks is a conflict of interest to weigh.

• Keys go to third-party relays. Using any relay means sending your prompts and code through that service. The "local only" claim covers Switch itself, not the relay you route through.