Building your own AI machine

What you’ll learn

How to choose the parts for a local AI machine and put them together, even if you have never built a PC before, with specific guidance on the two choices that matter most: the graphics card and the memory.

It is more approachable than it looks

Building a PC has a reputation for being difficult, and it really is not. The parts only fit together one way, each comes with a manual, and there is nothing inside that a careful afternoon cannot handle. It is closer to assembling flat-pack furniture than to electronics repair. The reward is a machine tuned to what you want, usually at better value than a pre-built one, and the understanding to upgrade it later.

For local AI, one part dominates every decision. Get the graphics card right, and the rest is about supporting it well.

The graphics card

The GPU is the single most important choice, and the number that matters is VRAM, the memory on the card, because it sets how large a model you can run. More VRAM beats a faster card with less of it, almost every time.

Some concrete starting points, which our reference builds cover in full:

• Best value: a used RTX 3090. Its 24GB often costs less than a new 16GB card, and 24GB runs models up to around 32B. If you are comfortable buying second-hand, little else competes.
• New, mid-range: an RTX 5080 is fast but has 16GB, while an RTX 4060 Ti 16GB is a cheaper route to the same memory.
• Flagship: an RTX 5090, with 32GB and the speed to match.
• The AMD option: a Radeon RX 7900 XTX offers 24GB for less than NVIDIA, at the cost of a less mature software stack. Expect a little more troubleshooting.

A note on NVIDIA versus AMD: NVIDIA’s CUDA is the path of least resistance, and almost every tool targets it first. AMD works and is cheaper for the memory, but you will occasionally pay for that in setup friction. For a first build, NVIDIA keeps things simple. And if you would rather not build at all, a high-memory Mac, covered in GPUs and Apple Silicon, is the no-assembly alternative.

Memory (RAM)

System RAM is separate from the GPU’s VRAM, and both matter. RAM holds the operating system, loads models before they reach the GPU, and takes the overflow if you run a model partly on the processor.

A good rule is to have at least as much system RAM as your card has VRAM, and preferably more:

• 32GB is a sensible floor for a serious build.
• 64GB is comfortable, and what we would suggest for most people.
• 128GB is worth it if you plan to offload very large models partly to the processor, or to run models that do not fit in VRAM alone.

Use current-generation DDR5. Memory speed makes little difference to GPU inference, so do not overpay for the fastest kits; capacity is what counts.

The supporting parts

• Power supply. This is the part people most often under-spec, and it matters for safety as well as stability. A high-end card like the RTX 5090 draws around 575W, so pair it with a quality 1000W or larger unit with the correct connectors. Buy a reputable supply with headroom; it is not the place to save money.
• Storage. Models are large, tens of gigabytes each, so use a fast NVMe SSD and start with 1TB to 2TB. It fills faster than you expect once you start collecting models.
• Processor. For GPU inference the CPU does not need to be top-tier; a modern mid-range chip is plenty. It matters more if you offload work to it, or run models on the CPU alone.
• Motherboard. Match it to your CPU and RAM, and check it has the slots you need. If you might add a second GPU later, look for a board with the spacing and PCIe slots for it.
• Case and cooling. Modern graphics cards are physically large, so check the card fits your case before buying both. Good airflow keeps a hard-working GPU cool and quiet.

Putting it together

The build itself is methodical. In rough order: fit the processor and memory onto the motherboard, mount the board in the case, install the power supply, seat the graphics card in the top PCIe slot, connect the power cables, and tidy the wiring. Work gently, follow the motherboard manual, and there is little to fear. Plenty of step-by-step build videos exist for whichever parts you choose, and following one for your first build is time well spent.

Once it powers on, install the operating system and, on an NVIDIA card, the current drivers with CUDA. After that, Ollama will have you running a model within minutes.

What can go wrong

• An underpowered power supply. The most common and most consequential mistake. Size it for your GPU with headroom.
• A case too small for the card. Measure. Big graphics cards do not fit every case.
• Too little RAM. Skimping here shows up when loading models and when offloading to the CPU. Follow the rule above.
• Forgetting there is no built-in display output. Many processors have no integrated graphics, so nothing appears on screen until the GPU is installed and its drivers are in. That is normal, not a fault.
• Mismatched parts. Check the processor socket, the RAM generation, and the case clearance all agree before you buy.

Next steps

For complete parts lists at several price points, see the reference builds. Once your machine is running, the hardware matrix shows what it can run, and the cost calculator helps you weigh the build against paying for an API.