Choosing your first model
There are thousands of models and it is easy to freeze at the choice. Here is a short, practical way to pick a good first one for your hardware and your task.
What you’ll learn
How to narrow thousands of models down to one sensible first choice, based on the two things that actually matter at the start: what your hardware can run, and what you want it to do.
The concept
Choosing a model feels overwhelming because there are so many, but most of them are variations you can ignore at first. Two questions cut through almost all of it.
What will fit? A model has to fit in your memory to run well. The rough guide is that a model at a 4-bit quantisation needs a bit more than half a gigabyte of memory per billion parameters, so an 8B model wants around 5GB, and a 32B model around 20GB. Our hardware matrix does this properly: tell it your VRAM or unified memory and it shows what fits.
What is it for? A general model handles chat, writing, and summarising. A code-specialised model is better at programming. There are also models for images, speech, and embeddings. Pick the type that matches your main use, and do not expect one model to be best at everything.
Doing it
A reliable path for a first model:
- Find your memory budget. Note your GPU’s VRAM, or your Mac’s unified memory. Leave a couple of gigabytes spare for the system and context.
- Pick the largest general model that fits comfortably, rather than the biggest one that technically loads. A smaller model with room to breathe beats a larger one that is starved of memory for context.
- Start with a well-supported model. For a first run, something like Llama 3.2 3B on a laptop, or Qwen3 8B on a 12GB card, gives a good experience with wide tooling support. If your interest is coding, Qwen2.5-Coder 7B is a sensible start.
- Run it with a tool that manages the details, such as Ollama, so you are not wrestling with files on day one.
What can go wrong
- Reaching for the biggest model you can technically load. It will run out of memory once the conversation grows, or run painfully slowly. Size down.
- Judging a small model by frontier standards. A 3B model is genuinely useful, but it is not a frontier model. Match your expectations to its size.
- Picking a model with little tooling support. A well-supported model saves you hours. The catalogue notes which formats and tools each one works with.
Next steps
Once you have a model running, it is worth understanding quantisation, which is what lets these models fit on ordinary hardware, and browsing the model catalogue to see what else fits your machine.