DeepSeek releases V4 Preview with a hybrid-attention redesign
Originally announced by DeepSeek . We link the primary source so you can read it for yourself.
DeepSeek has released a preview of DeepSeek V4, described not as an iteration on its previous generation but as an architectural reset centred on a hybrid approach to attention. Coming after the R1 reasoning model and the V3 line, it signals the lab is willing to change the underlying design rather than only scale what it had.
What was announced
The headline is architectural. Where much of the field has scaled existing designs, DeepSeek frames V4 around a reworked attention mechanism intended to handle long context more efficiently. As a preview release, the specifics are provisional and the performance figures are the lab’s own, so they warrant the same caution we apply to any developer-reported results.
Why it matters
Attention is the part of a transformer whose cost grows sharply with context length, so genuine efficiency gains there matter a great deal for running long-context workloads locally. If the approach holds up under independent testing, it could ease one of the real constraints on local inference. If it does not, it will be an interesting experiment. A preview is the right time to be curious and not yet convinced.
The context
DeepSeek has been one of the more consequential open-weight labs, in part because its R1 reasoning model was released under a permissive MIT licence at a capability level that surprised many. A V4 built on a new architecture is worth watching for what it says about where efficient long-context inference is heading, as much as for the model itself.
We will assess V4 for the catalogue once a full release and independent evaluations are available.