Choosing a local coding model

What you’ll learn

How to choose a coding model that fits your hardware and your workflow, and why the best model for autocomplete is not the same as the best one for an agent.

Match the model to the job

Different coding tasks want different things:

• Autocomplete needs to be fast above all. A small model gives quick, low-latency suggestions. Quality matters less than speed here, because you are the one deciding what to accept.
• Chat and multi-file edits benefit from a larger, more capable model, since it is reasoning about your code rather than finishing a line.
• Agentic work, where the tool explores a codebase and makes changes across files, wants a model trained for it, with strong tool use and a long context window.

Many people run two models: a small one for autocomplete, and a larger one for chat and edits. Tools like Continue let you assign each separately.

By hardware

• Around 12GB of VRAM. Qwen2.5-Coder 7B at a good quantisation is a strong, well supported choice, and small enough to double as an autocomplete model.
• A 24GB card. Qwen3-Coder 30B-A3B is a capable agentic coding model that fits at 4-bit, and its mixture-of-experts design keeps it quick. For agent-driven work specifically, Devstral Small was built for it.
• Less than 12GB. Smaller coding models still help with autocomplete and simple edits, though you will feel the size on harder tasks.

Doing it

1. Work out your memory budget, and check what fits with the hardware matrix.
2. Pick a coding model in that budget using the guide above.
3. If you have room, set up two models: a small fast one for autocomplete and a larger one for chat and edits.
4. Try it on your own code. Benchmarks are a starting point, not a verdict. The only test that matters is whether it helps on your actual work.

What can go wrong

• Using one heavy model for autocomplete. Suggestions lag, and you stop using them. Keep autocomplete on a small, fast model.
• Choosing on benchmarks alone. A model that tops a leaderboard may not suit your language or style. Trial two finalists on real tasks.
• Overreaching your memory. A coding model crammed into VRAM with no room for context will drop the thread on multi-file work.

Next steps

With a model chosen, the natural next step is local coding agents, for when you want the tool to do more than suggest. The coding tools catalogue lists the editors and assistants to pair with your model.