Multi-user access and gateways

What you’ll learn

The layer that sits between your served models and the people using them: gateways for routing and access control, a front end for non-technical users, and observability so you can see what is going on.

The gateway

Once you serve more than one model, or want more than raw access to a single one, a gateway becomes useful. LiteLLM is the common choice: it puts one OpenAI-compatible endpoint in front of many models, local and remote, and adds the things a shared deployment needs:

• Routing across models, so applications ask for a capability and the gateway decides which model serves it.
• Access control through API keys, so you can grant, revoke, and scope access per user or team.
• Budgets and cost tracking, so usage is visible and controllable, which matters as soon as more than one person is involved.

The value is that your applications and users talk to one stable endpoint, while you change what runs behind it.

A front end for people

Not everyone wants an API. Open WebUI is a common self-hosted, ChatGPT-style front end that gives a team accounts, chat, document tools, and shared workspaces over your served models. It turns a raw endpoint into something a non-technical colleague can use. Note its licence includes a branding-preservation clause in recent versions, worth reading if you plan to rebrand it.

Seeing what happens

A shared deployment you cannot observe is one you cannot run well. Langfuse provides tracing, cost and latency tracking, and evaluation, and it integrates with a LiteLLM gateway so every call is logged. When a user reports a bad answer or a bill looks wrong, this is how you find out why. It is self-hostable, so the traces stay on your infrastructure.

Doing it

1. Put a gateway in front of your served models, starting with LiteLLM, and issue keys per user or team.
2. Add a front end like Open WebUI for people who want a chat interface rather than an API.
3. Wire in observability with Langfuse so you can see usage, cost, and failures.
4. Set budgets and access deliberately, before rather than after usage grows.

What can go wrong

• No access control. An open endpoint on a network is a liability. Put keys in front of it from the start.
• No visibility. Without tracing, you cannot debug bad answers or explain costs. Add it early, not after the first problem.
• Reinventing the front end. A team rarely needs a custom UI. An existing self-hosted front end saves a lot of effort.

Next steps

For the machines underneath all of this, see planning hardware for a team. The orchestration and monitoring categories in the catalogue cover the tools here.