Ollama's August update speeds up Apple Silicon inference with speculative decoding
Originally announced by Ollama . We link the primary source so you can read it for yourself.
Ollama has published version 0.32.6, and while it is a point release rather than a landmark, a couple of the changes are worth knowing if you run models locally, particularly on a Mac.
The news
The headline change is faster generation on Apple GPUs. On Apple Silicon, Ollama’s MLX engine now uses a model’s own multi-token-prediction head to drive speculative decoding automatically, rather than needing a separate draft model configured by hand. In the same release, draft-model output heads are quantised at the requested type when building the speculative-decoding drafts.
Alongside that, the update aligns Ollama’s streaming responses with OpenAI’s wire format, which makes it a cleaner drop-in for tools that expect that format, and widens hardware support, including CUDA on Windows ARM64 and newer NVIDIA parts, with lower memory use reported on some Linux integrated GPUs. There are also fixes to the terminal interface.
Why it matters
Speculative decoding is a technique for speeding up generation: a smaller, faster draft proposes several tokens, and the main model checks them in one pass, so more tokens are produced per step when the guesses are good. Doing this automatically, using the model’s own prediction head rather than a hand-configured second model, removes a piece of setup that most people never bothered with, and it lands on Apple Silicon, where a large share of local users run their models. The gain depends on the model and the workload, so treat it as a useful speed-up rather than a fixed figure.
The move to match OpenAI’s streaming format is quieter but practical. A great deal of local tooling is written against that format, so closer compatibility means fewer rough edges when pointing an application at a local model instead of a hosted one.
The history
Ollama has become the common on-ramp to running models locally by wrapping engines like llama.cpp and MLX behind a simple interface, so improvements to the engines it bundles reach a wide audience without anyone changing how they work. This release is a good example: mixture-of-experts models such as Qwen3 30B-A3B benefit from the faster decoding without any change on the user’s part.
Worth a caveat
This is an incremental release, and we would not want to overstate it. If you are not on Apple Silicon and not building against the OpenAI streaming format, there may be little here that changes your day. For the many people who are, it is a small, welcome improvement of exactly the kind that keeps local inference getting quietly faster.