Skip to content
local-ai

Qwen2.5-Coder 7B Instruct

Alibaba · Code · 7B · 131k context · Released 12 November 2024

Commercial use permitted Open weights Runs on CPU Apple Silicon

Alibaba's code-specialised 7B model, strong at code completion and generation well beyond what its size would suggest. A practical choice for a local coding assistant on a mid-range GPU, and permissively licensed.

Strengths

  • Genuinely capable at code for its size
  • Apache 2.0, so no commercial-use conditions to reason about
  • Runs comfortably on a 12GB card at a good quantisation

Weaknesses

  • A 7B model still trails larger models on complex, multi-file reasoning
  • General chat and non-code tasks are not its focus
  • Long-context code understanding degrades well before the 128k ceiling

Hardware requirements

QuantisationApprox. VRAMNotes
Q4_K_M~4.7GBCommon default, fits an 8GB card with short context
Q8_0~8GBNear-lossless, a good fit for 12GB cards
FP16~15GBFull precision, needs 16GB or more

Also runs on CPU (slower). Optimised builds available for Apple Silicon.

What you'd need to run this

Roughly what a machine to run this would need, at up to three levels of quality. Memory is the deciding factor.

Minimum to run it

Q4_K_M · ~4.7GB needed

One 12GB GPU

NVIDIA GeForce RTX 3060 12GB

or a Mac or mini-PC with unified memory, if you prefer no discrete GPU, Mac mini M5 Pro .

At least 32GB of system RAM alongside the card.

around £700–£1,100

What else 12GB runs →

For good quality

Q8_0 · ~8GB needed

One 12GB GPU

NVIDIA GeForce RTX 3060 12GB

or a Mac or mini-PC with unified memory, if you prefer no discrete GPU, Mac mini M5 Pro .

At least 32GB of system RAM alongside the card.

around £700–£1,100

What else 12GB runs →

Best quality

FP16 · ~15GB needed

One 24GB GPU

NVIDIA Tesla P40

or a Mac or mini-PC with unified memory, if you prefer no discrete GPU, Mac mini M5 Pro .

32–64GB of system RAM alongside the card.

around £700–£1,100

What else 24GB runs →

Licence

Apache 2.0 — read the licence

Benchmarks

BenchmarkScoreSourceAs of
MBPP83.5 Qwen2.5-Coder technical report November 2024

How it compares

How this model’s reported scores sit against other models we cover, on the same benchmarks. This model is highlighted.

MBPP

higher is better

Qwen2.5-Coder 7B Instruct: common questions

What hardware do I need to run Qwen2.5-Coder 7B Instruct?
At its most compressed (Q4_K_M) it needs roughly 4.7GB of VRAM, and about 8GB for good quality. VRAM figures are approximate and depend on context length and settings.
Is Qwen2.5-Coder 7B Instruct free for commercial use?
Yes. Qwen2.5-Coder 7B Instruct is licensed under Apache 2.0, which permits commercial use with no meaningful conditions.
Can I run Qwen2.5-Coder 7B Instruct on Apple Silicon?
Yes. Qwen2.5-Coder 7B Instruct has builds optimised for Apple Silicon, through MLX or GGUF on a Mac.
Does Qwen2.5-Coder 7B Instruct run on CPU?
Yes, Qwen2.5-Coder 7B Instruct can run on the CPU, though generation is slower than on a GPU.
What is Qwen2.5-Coder 7B Instruct's context window?
Qwen2.5-Coder 7B Instruct has a context window of 131,072 tokens, about 131k.

Availability

Where to get quantised weights

Some of the best quantised weights are made by the community, not the model’s authors. Look this model up on these providers:

  • Bartowski GGUF Q2-Q8 (imatrix)

    A wide, reliable range of imatrix GGUF quants, typically Q2 through Q8.

  • Unsloth GGUF (Dynamic 2.0, imatrix)

    Dynamic and imatrix GGUF quants that often hold quality better than a plain quant at the same bit-width, especially at 4-bit and below.

  • MLX community MLX 4-bit and 8-bit

    MLX quants for Apple Silicon, usually 4-bit and 8-bit.

Recommended for

  • Local code autocomplete and generation on a 12GB card
  • A permissively licensed coding model for commercial work
  • Pairing with an editor tool like Continue or Aider

Related models

Run it with

Related guides

Glossary

Catalogue entry last verified 30 July 2026. Specifications change; verify anything you are about to spend money on.