Self-hosting at scale: team, business, or enterprise

What you’ll learn

The rough tiers of self-hosting at organisational scale, how to tell which one you are actually in, and why the honest answer for most organisations is smaller than they expect.

Local does not mean small

When we say local, we mean self-hosted, on hardware you control. That spans an enormous range. At one end is a colleague running a model on a workstation. At the other is an enterprise serving a model beyond 100 billion parameters to thousands of users from its own cluster. Both keep the weights and the data in-house, and both are local AI. But they are wildly different engineering problems, and treating them as the same is how projects go wrong.

It helps to think in three tiers.

The three tiers

Individual and small team. A few people, one GPU or a capable Mac, models up to around 32B. This is the ground the rest of this site mostly covers: pick a model, run it with Ollama or llama.cpp, share it lightly if at all. A Qwen3 32B on a 24GB card serves a small team perfectly well for many tasks.

Business and team scale. A handful to a few dozen users, or steady automated workloads, served from one or a few GPUs on a server. Here you move from running a model to serving it: a proper serving engine, a gateway for access and cost control, a front end for people, and hardware sized for concurrent use. The next three guides, starting with serving models at scale, cover this tier. Most organisations that adopt local AI live here, and can do so without exotic infrastructure.

Enterprise and server-class. Many users or heavy automated demand, models beyond 100 billion parameters, and hardware that spans multiple GPUs or multiple machines. At this tier the model no longer fits on one card, so you are into parallelism across GPUs, fast interconnect between them, and the networking, power, and cooling that a cluster needs. A model like gpt-oss-120b is the accessible entry, running on a single 80GB GPU, while something like Kimi K3 sits at the far end, needing a cluster. The last two guides in this topic cover this tier.

How to tell which you are

The tier is set by your workload, not your company’s size. Three questions decide it:

• Concurrency. How many requests at once, at peak? A few is team scale; hundreds is enterprise.
• Model size. Does the model you need fit on one GPU, or does it need several? That single fact changes everything below it.
• Reliability. Is this a helpful tool, or a system with uptime commitments? The latter pulls you up a tier regardless of the other two.

The honest part

The jump from business scale to enterprise is not incremental. It is a step change in cost, complexity, and the skills required, from configuring a serving engine to running distributed systems on specialised hardware. Most organisations adopting local AI are at business or team scale and should stay there deliberately, rather than building enterprise infrastructure for a workload that does not need it. Equally, a genuine enterprise workload run on a team-scale setup will buckle. Matching the tier to the need, honestly, is the whole game.

Next steps

If you are at business or team scale, continue to serving models at scale. If your model does not fit on a single GPU, or you are serving heavy demand, see running large open models and enterprise hardware and networking.