Does a newer model mean a better one?

“There is a new version, so it must be better” is one of the most natural assumptions in local AI, and one of the least reliable. Sometimes a new release is a genuine step up. Sometimes it moves almost nothing. Sometimes it is worse at the specific thing you relied on. And sometimes, as we will see, there is no newer model at your size at all. This guide is about telling those cases apart.

How much a version step actually buys

Start with the honest baseline: a minor or patch bump usually changes very little. The independent analyst Artificial Analysis notes that most models move by less than a point on its Intelligence Index between patch versions. A larger generational step can be substantial, but “how much” is genuinely hard to measure, for a reason the previous guide sets out: the trackers keep changing their benchmark mix (Artificial Analysis has added and dropped benchmarks across index versions as older ones saturate), so a headline score that “went up” or “fell” between versions is often not comparing like with like. When you read “version N scores X”, check whether version N-1 was scored on the same tests.

So the first answer is deflationary: for everyday work, a new minor version is frequently not worth the download.

Which parts get better, and which do not

When models do improve, the gains are uneven. Through 2026 the improvements have concentrated in maths, then broader reasoning, then coding and agentic tool use; Epoch AI reports that maths benchmarks saturated first once reasoning models arrived, while a 2025 survey (arXiv 2511.01365) finds coding and long-horizon tasks remain comparatively hard. Meanwhile factual reliability, instruction-following, formatting and tone can sit flat, or move backwards, in the very same release. “Smarter” and “more reliable” are not the same axis and do not always move together: OpenAI’s own system card for o3 and o4-mini shows those newer reasoning models hallucinating more than the older o1 on a factual-recall test (PersonQA, lower is better: o1 0.16, o3 0.33, o4-mini 0.48). A model can get better at competition maths and worse at not making things up, in one step.

Yes, models sometimes get worse

This is the part people find surprising, so it is worth being concrete, because it is measured rather than anecdotal.

The clearest study is a three-month comparison of GPT-4 and GPT-3.5 (Chen, Zaharia and Zou, 2023). Between March and June 2023, the same deployed GPT-4 went from 84% to 51% at identifying prime numbers, from 52% to 10% at producing directly executable code, and from near-perfect to almost zero on one instruction-following measure, while at the same time improving at multi-hop reasoning. The authors are careful that some of the code drop was extra formatting around the answer rather than lost logic, and that the causes can only be inferred because model updates are opaque. Both caveats are part of the lesson: a “regression” is often a change in behaviour or output format, not a loss of underlying ability, and you frequently cannot see why.

Regressions reach shipped products. In April 2025 an update to GPT-4o made it noticeably sycophantic, agreeing with users and flattering them; OpenAI rolled it back within days and explained that the training had placed too much weight on short-term user approval signals. A newer version, shipped to millions, was worse, and had to be withdrawn.

Why a newer model can be worse

Several mechanisms are well documented, and they are worth knowing because they tell you what to check:

• Reward over-optimisation. Tuning hard on a preference signal (such as thumbs-up votes) can trade away other qualities, as the GPT-4o sycophancy case shows.
• The safety tax. Adding safety alignment can measurably reduce capability; one study is titled, plainly, “Safety Tax: Safety Alignment Makes Your Large Reasoning Models Less Reasonable”. More refusals and more caution can read as “got worse” on a capability test.
• Benchmark overfitting. Apparent gains can be the test leaking into training rather than real ability. When researchers built a fresh copy of a grade-school maths benchmark (GSM1k), some model families dropped by around 13 percentage points, a sign they had partly memorised the original.
• Prompt and template sensitivity. This one matters most for the “my new version got worse” complaint: formatting changes alone can swing a model’s score by up to 76 points in few-shot tests. A new chat template or default system prompt between versions can break behaviour you depended on without any change to the weights at all.
• Quantisation, the local-specific one. For local AI, a “new version” is often not new weights but a different quantisation of the same model. A heavier quant trades quality for size, and an aggressive one (below 4-bit) degrades noticeably. If a model “got worse” after you re-downloaded it, check whether you changed the quant before you blame the model. A Q4 and an FP16 build are different subjects.

A worked example: the Qwen 4B line

A concrete case pulls these threads together, and it is a counterintuitive one. Suppose you run a small Qwen model, say the 4-billion or 9-billion-parameter size, and you reasonably ask: is there a newer generation I should move to?

The answer, as of late 2026, is that there is not. Alibaba shipped small dense models at 0.8B, 2B, 4B and 9B in the Qwen3.5 line (early 2026, Apache 2.0). The generations that followed did not ship a 4B or a 9B dense model at all: Qwen3.6 and Qwen3.8 moved their small end up to a 27B dense model and to mixture-of-experts designs, and Qwen3.7 appeared only as hosted, closed tiers. So “use the latest” is not even an option here: at 4B and 9B, the Qwen3.5 models are the current ones, not superseded ones. A family’s small-dense tier can simply be left behind when its frontier moves to larger and sparser models.

The same example shows the upside of a real generation step, when it happens. Artificial Analysis found the Qwen3.5 small models roughly doubled the previous Qwen generation’s score at the same sizes (its Intelligence Index put the 9B at 32 versus 17 for the prior 8B, and the 4B at 27 versus 18). When a genuine generational jump lands at a size you run, it can be worth taking. And the same source supplies the caution that belongs on every small-model recommendation: on a hallucination-sensitive test, these 4B and 9B models fabricated answers around 80% of the time. A small model knows far less than its benchmark scores suggest, whatever its version number.

So how much does “the latest” matter?

Put the pieces together and a practical rule falls out.

For simple, high-volume work (lookups, classification, extraction, latency-bound interfaces), the newest or largest “reasoning” model is often the wrong tool, not just an unnecessary one: research on over-thinking finds reasoning models can be five to twenty times slower on simple tasks with no meaningful gain. A cheaper or older model is the better choice there, and an older version can be genuinely more reliable on a specific axis, as the o1-versus-o3 factuality numbers show.

For hard, multi-step problems with a checkable answer, a real generational step is where the gains concentrate, so the latest is more likely to earn its place.

Is progress slowing overall? That is genuinely contested. Some analyses point to plateauing scores and rising cost per point of accuracy; Epoch’s own data argues the opposite, that several hard metrics are still accelerating. The honest position is that it depends entirely on which benchmark you watch, and that saturated tests make recent gains look smaller than they are.

The reliable move is the one this whole topic keeps returning to: do not take a version number, or a leaderboard delta, as proof. Decide what you actually need the model to do, and test the candidates on your own material. It is the only way to tell a real improvement from a re-weighted benchmark, a formatting change, or a quant you downloaded by mistake. It is also why our model recommendations lead with the newest option but keep strong older models as labelled alternates: newer is the right default, not a guarantee.