Llama 3.2 3B Instruct
Meta · Text generation · 3B · 131k context · Released 25 September 2024
A small 3-billion-parameter model that runs on almost anything, including laptops without a dedicated GPU. Capable for its size at summarising, rewriting, and simple assistant tasks, and a good first model to try.
Strengths
- Runs on laptops and even CPU-only machines at usable speed
- Low memory footprint leaves room for long context
- A gentle introduction to running models locally
Weaknesses
- Limited reasoning depth, it will struggle with harder tasks
- More prone to factual mistakes than larger models
- Same Llama licence conditions as its larger siblings
Hardware requirements
| Quantisation | Approx. VRAM | Notes |
|---|---|---|
| Q4_K_M | ~2.2GB | Runs on almost anything, including CPU |
| Q8_0 | ~3.4GB | Near-lossless and still very light |
| FP16 | ~6.5GB | Full precision, comfortable on an 8GB card |
Also runs on CPU (slower). Optimised builds available for Apple Silicon.
What you'd need to run this
Roughly what a machine to run this would need, at up to three levels of quality. Memory is the deciding factor.
Minimum to run it
Q4_K_M · ~2.2GB needed
One 12GB GPU
NVIDIA GeForce RTX 3060 12GBor a Mac or mini-PC with unified memory, if you prefer no discrete GPU, Mac mini M5 Pro .
At least 32GB of system RAM alongside the card.
around £700–£1,100
What else 12GB runs →For good quality
Q8_0 · ~3.4GB needed
One 12GB GPU
NVIDIA GeForce RTX 3060 12GBor a Mac or mini-PC with unified memory, if you prefer no discrete GPU, Mac mini M5 Pro .
At least 32GB of system RAM alongside the card.
around £700–£1,100
What else 12GB runs →Best quality
FP16 · ~6.5GB needed
One 12GB GPU
NVIDIA GeForce RTX 3060 12GBor a Mac or mini-PC with unified memory, if you prefer no discrete GPU, Mac mini M5 Pro .
At least 32GB of system RAM alongside the card.
around £700–£1,100
What else 12GB runs →Licence
Llama 3.2 Community License — read the licence
Llama 3.2 3B Instruct: common questions
- What hardware do I need to run Llama 3.2 3B Instruct?
- At its most compressed (Q4_K_M) it needs roughly 2.2GB of VRAM, and about 3.4GB for good quality. VRAM figures are approximate and depend on context length and settings.
- Is Llama 3.2 3B Instruct free for commercial use?
- Commercial use is permitted, but with conditions. Commercial use is permitted, with the same 700 million monthly active user threshold as other recent Llama releases above which a separate licence from Meta is required. Read the licence before relying on it at scale.
- Can I run Llama 3.2 3B Instruct on Apple Silicon?
- Yes. Llama 3.2 3B Instruct has builds optimised for Apple Silicon, through MLX or GGUF on a Mac.
- Does Llama 3.2 3B Instruct run on CPU?
- Yes, Llama 3.2 3B Instruct can run on the CPU, though generation is slower than on a GPU.
- What is Llama 3.2 3B Instruct's context window?
- Llama 3.2 3B Instruct has a context window of 131,072 tokens, about 131k.
Availability
- Official page
- Hugging Face
- ollama run llama3.2:3b
Where to get quantised weights
Some of the best quantised weights are made by the community, not the model’s authors. Look this model up on these providers:
- Unsloth GGUF (Dynamic 2.0, imatrix)
Dynamic and imatrix GGUF quants that often hold quality better than a plain quant at the same bit-width, especially at 4-bit and below.
- Bartowski GGUF Q2-Q8 (imatrix)
A wide, reliable range of imatrix GGUF quants, typically Q2 through Q8.
- MLX community MLX 4-bit and 8-bit
MLX quants for Apple Silicon, usually 4-bit and 8-bit.
Recommended for
- A first local model on a laptop
- Summarising and rewriting where a small model is enough
- Running alongside other software with memory to spare
Related guides
Glossary
Catalogue entry last verified 30 July 2026. Specifications change; verify anything you are about to spend money on.