LiteLLM

What it is

LiteLLM is an MIT-licensed, self-hostable AI gateway. The page is vendor marketing, so every figure below is the vendor's own claim. Its main value for a security review is the split between what the open-source tier includes and what needs the paid tier, plus the checkable supply-chain claims.

Preconditions / Affected

Open source vs Enterprise. The page's plan lists put these in the open-source tier:

• 140+ providers
• Langfuse, Arize Phoenix, LangSmith and OTEL logging
• Virtual keys, budgets and teams
• Load balancing and RPM/TPM limits
• LLM guardrails

The page says Enterprise "adds SSO, RBAC, audit logs, and support on top, not a different core", and lists JWT auth, SSO and audit logs under the paid plan. SCIM appears only in the security-controls list, with no tier stated. If your review requires SSO, RBAC or an audit trail, assume a paid license until the docs confirm otherwise.

Runtime dependencies. It runs on your own Postgres and Redis. Deployment options are an official Helm chart, a Terraform module, or Docker images (database-bundled and non-root variants).

What to do

Checks worth running before adoption, based on what the page claims:

• Verify the Cosign signature on the image before deploy. The page says images are signed and hardened.
• Run your own scan. The page claims Grype shows zero high or critical findings and that CodeQL covers the codebase.
• Confirm the "no telemetry" claim by inspecting egress from a test deployment.
• Read the public changelog and security disclosures. The page says fixes and disclosures are published, and it describes a separate stable release track whose production images ship after load testing. Pin to that track.
• Test the budget enforcement. The page says hard budgets exist per key, team, org and model, with daily and monthly resets, and that requests stop at the cap. Rate limits and leaked-key protection are listed as separate controls.
• Wire secrets to AWS Secrets Manager, Vault or Azure Key Vault. The page says keys are never hardcoded.

Caveats

• Benchmark scope. The Rust gateway's 0.66 ms p99 overhead (3.5× lower than "the next AI gateway") was measured on identical hardware against a deterministic mock upstream with a single client. It measures gateway overhead only, not behavior under concurrent load with real providers. The page says scripts and charts are public, so rerun it. It does not say whether the Rust gateway has the same feature set as the main gateway, or whether the signed images, scanning and guardrails claims cover it.
• Guardrails. Presidio and Lakera are named, with "and more". The page gives no detection-quality data.
• Customer quotes. The testimonials and the "costs fell by as much as 56%" claim are unsourced on the page.
• Air-gapped use. The page claims it is supported, but gives no details on how guardrails or model access work there.