Getting started with local AI coding

What you’ll learn

The main kinds of local AI coding tool, how a local setup fits together, and how to get one working, along with an honest sense of where it stands against cloud assistants.

Being honest up front

Local coding models have come a long way, and for many everyday tasks they are genuinely useful. They are also, at the sizes most people can run, still behind the best cloud coding models on the hardest, most sprawling work. If your priority is raw capability and you are comfortable sending code to a provider, a cloud assistant will often do more.

Local coding earns its place when the code must stay on your machine, when you want to work offline, when you would rather not depend on a subscription, or when you simply want to. Plenty of people run local models for the routine work and reach for the cloud on the occasional hard problem. That is a perfectly good way to work.

The kinds of tool

Local coding tools fall into a few groups:

• Autocomplete and chat in your editor. Extensions like Continue add inline suggestions, a chat panel, and multi-file edits inside VS Code or JetBrains, pointed at a local model.
• Terminal assistants. Aider pairs with your git repository from the command line, making changes as reviewable commits.
• Coding agents. Tools like Cline and OpenHands take on larger tasks more autonomously, planning and acting with access to your files and terminal. These are covered in local coding agents.
• AI-first editors. Zed builds AI features into a fast native editor.

Doing it

A dependable first setup:

1. Install a model runner. Ollama is the easiest, and it exposes a local API that coding tools understand.
2. Pull a coding model that fits your hardware, such as Qwen2.5-Coder 7B on a 12GB card. Choosing one is covered in choosing a local coding model.
3. Install a tool and point it at Ollama. Continue is a good first choice, and lets you use a small fast model for autocomplete and a larger one for chat.
4. Set a sensible context length for your hardware. Code tasks benefit from a long context window, but it costs memory, so match it to what your machine can hold.

What can go wrong

• Expecting cloud-level results from a small local model. Judge it for what it is. A 7B model is a capable assistant, not a senior engineer.
• Starving the model of context. Set the context length too low and the model loses track of your files; set it too high for your memory and it slows or crashes. Find the balance for your hardware.
• Slow autocomplete. Inline suggestions need to be fast to be useful. Use a small, quick model for autocomplete even if you use a larger one for chat.

Next steps

Pick a model with choosing a local coding model, then, when you want a tool that does more than suggest, read about local coding agents.