LLM Gateway
LLM Gateway is an open-source API gateway for Large Language Models (LLMs). It acts as a middleware between your applications and various LL
On LLM Gateway
LLM Gateway unifies 40+ LLM providers behind a single API. It lets you route requests, switch models, and track costs without changing code. The service supports OpenAI, Anthropic, Google, and dozens more, plus self-hosting and SOC 2 Type II compliance. It’s aimed at teams that want to avoid vendor lock-in while keeping latency and spending under control.
LLM Gateway is the pragmatic choice if you want to escape vendor lock-in without rewriting code or paying hidden markups. Its BYOK tier is genuinely free with solid analytics, and the self-host option gives teams full control. Enterprise features like SSO and SOC 2 Type II make it viable for regulated environments. The only real downside is the 5% fee on the credits plan, which still undercuts many competitors.
Key features
- Unified API compatible with OpenAI SDKs
- Multi-provider support across 40+ providers
- Real-time cost and latency analytics
- Project-level usage explorer
- Per-model/provider spend breakdown
- Secure key management dashboard
- LLM guardrails (prompt injection, PII detection)
- Automatic failover for 99.9999% uptime
- Enterprise SSO and audit logs
- SOC 2 Type II certification
Pros and cons
- Unified API that works with existing OpenAI SDKs (change base URL only)
- Access 40+ providers and 200+ models without vendor lock-in
- Real-time cost and latency analytics with per-model/provider breakdowns
- Bring Your Own Keys option is free with routing and analytics included
- Self-hosted option under AGPLv3 for full control
- SOC 2 Type II compliant and supports enterprise SSO
- Automatic failover across providers for 99.9999% effective uptime
- LLM guardrails for prompt injection, PII detection, and malicious requests
- Credits plan adds a 5% platform fee on top of provider rates
- Self-hosting requires operational overhead and infrastructure management
- Not all niche models are available; coverage depends on provider list
- Enterprise tier pricing is custom and can be expensive for small teams
Pricing
- Route requests across 40+ providers
- Real-time cost and latency analytics
- Secure key management dashboard
- OpenAI SDK compatibility (change base URL)
- Project-level usage explorer
- Per-model/provider spend breakdown
- Errors and reliability monitoring
- LLM guardrails (prompt injection, PII detection)
- SOC 2 Type II compliance
- Route requests across 40+ providers
- Real-time cost and latency analytics
- Secure key management dashboard
- OpenAI SDK compatibility (change base URL)
- Project-level usage explorer
- Per-model/provider spend breakdown
- Errors and reliability monitoring
- LLM guardrails (prompt injection, PII detection)
- SOC 2 Type II compliance
- Self-hosted routing layer
- Open-source under AGPLv3
- Route requests across 40+ providers
- Secure key management dashboard
- LLM guardrails (prompt injection, PII detection)
- Enterprise add-ons available
- Enterprise SSO (SAML & OIDC)
- Self-hosted or managed deployment
- Volume pricing and custom rate limits
- White-label gateway and chat playground
- Priority support
- SOC 2 Type II certified
- On-boarding assistance
- Dedicated shard option
Best for
Integrations & platforms
OpenAI SDK (Python, TypeScript, Next.js, cURL, Java, Rust, Go, PHP, Ruby) · Anthropic SDK · Vercel AI SDK · GitHub Copilot migration path · LiteLLM migration path · OpenRouter migration path · Portkey comparison page
Frequently asked questions
What makes LLM Gateway different from OpenRouter?
LLM Gateway offers full self-hosting under AGPLv3, deeper real-time cost and latency analytics, a Bring Your Own Keys free tier, and flexible enterprise add-ons such as dedicated shards and custom SLAs.
What models do you support?
Access 200+ models across 40+ providers including OpenAI, Anthropic, Google, Mistral, DeepSeek, Groq, AWS Bedrock, Azure, and more. See the full list at https://llmgateway.io/models.
What is your uptime guarantee?
With automatic failover across providers, the effective uptime is 99.9999%. Each provider averages ~94% uptime; the combined uptime across multiple providers reduces simultaneous downtime to near zero.
How much does it cost?
The Bring Your Own Keys tier is free with routing and analytics included. The Credits tier charges a 5% platform fee on top-ups. Enterprise pricing is custom with volume discounts, dedicated shards, and white-label options.
Can I self-host LLM Gateway?
Yes. LLM Gateway is AGPLv3-licensed and can be deployed on your own infrastructure for full control. Documentation and GitHub repo are available.
More in AI & ML
