LiteLLM
A gateway and SDK that gives you one OpenAI-compatible API across hundreds of model providers, local backends included. As a proxy it adds routing, rate limiting, cost tracking, and key management in front of your models.
Strengths
- One consistent API across local and remote models, easy to swap between them
- Proxy adds routing, budgets, cost tracking, and key management
- Works cleanly with local backends like Ollama and vLLM
Weaknesses
- Another service to run and maintain in front of your models
- Some advanced and enterprise features are gated
- Adds a layer that can complicate debugging
At a glance
- Licence
- MIT
- Pricing
- Open core
- Platforms
- Docker, Linux, macOS, Windows
- GPU required
- No
Links
Entry last verified 30 July 2026.