Skip to content
local-ai

Nemotron 3.5 Lightning

NVIDIA · Text generation · MoE 31.6B-A3.6B · 1000k context · Released 11 August 2026

Commercial use permitted Open weights

Nemotron 3.5 Lightning is the fast, distilled tier of NVIDIA's open Nemotron family, described by NVIDIA as distilled from its larger, unreleased Nemotron 3 Ultra. Two things make it notable for local AI. First, it is open in the fuller sense: the release includes the training data and recipes, not only the weights, which is rarer than open weights alone and matters for anyone fine-tuning or researching rather than only running a model. Second, it is a small-active mixture-of-experts, so it computes like a roughly 3.6B model while holding the breadth of a 30B one, which is what makes it cheap at the high request volumes agents generate. The practical caveat is runtime support: its hybrid Mamba-Transformer architecture runs surest on vLLM or TensorRT-LLM today, while the llama.cpp-based tools most people run locally are still catching up, so confirm your engine handles the architecture before assuming a GGUF build exists.

Strengths

  • Fully open, with weights, training data, and recipes under a permissive licence
  • Fast and cheap for its capability, with only about 3.6B parameters active per token
  • Very long context (one million tokens) and strong reported agentic performance

Weaknesses

  • Hybrid Mamba-Transformer support is still maturing in llama.cpp-based local runtimes
  • As a mixture-of-experts, all experts must be resident, so memory is set by the full ~31.6B, not the active 3.6B
  • Benchmark figures below are largely the developer's own claims, pending wider independent testing

Hardware requirements

QuantisationApprox. VRAMNotes
4-bit~18GBApproximate, a 30B mixture-of-experts at 4-bit. Runtime support for the hybrid architecture is still maturing, so confirm your engine handles it
8-bit~33GBHigher quality, needs 40GB or more
BF16~63GBFull precision, server or multi-GPU territory

What you'd need to run this

Roughly what a machine to run this would need, at up to three levels of quality. Memory is the deciding factor.

Minimum to run it

4-bit · ~18GB needed

One 24GB GPU

NVIDIA Tesla P40

or a Mac or mini-PC with unified memory, if you prefer no discrete GPU, Mac mini M5 Pro .

32–64GB of system RAM alongside the card.

around £700–£1,100

What else 24GB runs →

For good quality

8-bit · ~33GB needed

64GB of unified memory

Mac mini M4 Pro

or a single high-memory workstation card, NVIDIA RTX 6000 Ada Generation .

Unified memory is shared with the model, so it is already counted above.

around £1,100–£2,000

See the Affordable Mac build build What else 64GB runs →

Best quality

BF16 · ~63GB needed

96GB of unified memory

AMD Ryzen AI Max+ 395 (Strix Halo)

or an 80GB-class data-centre card, which is usually rented by the hour, NVIDIA A100 80GB .

Unified memory is shared with the model, so it is already counted above.

around £1,100–£2,000

See the Quiet large-model box build What else 96GB runs →

Licence

OpenMDW-1.1

Benchmarks

BenchmarkScoreSourceAs of
SWE-bench Verified51.56 NVIDIA (BF16) August 2026
GPQA Diamond75.44 NVIDIA (BF16) August 2026
MMLU-Pro81.94 NVIDIA (BF16) August 2026
Artificial Analysis Intelligence Index v4.1.124 Artificial Analysis (independent) August 2026
Terminal-Bench 2.124% Artificial Analysis (independent) August 2026

How it compares

How this model’s reported scores sit against other models we cover, on the same benchmarks. This model is highlighted.

SWE-bench Verified

higher is better

Terminal-Bench 2.1

higher is better

Artificial Analysis Intelligence Index v4.1.1

higher is better

GPQA Diamond

higher is better

MMLU-Pro

higher is better

Nemotron 3.5 Lightning: common questions

What hardware do I need to run Nemotron 3.5 Lightning?
At its most compressed (4-bit) it needs roughly 18GB of VRAM, and about 24GB for good quality. VRAM figures are approximate and depend on context length and settings.
Is Nemotron 3.5 Lightning free for commercial use?
Yes. Nemotron 3.5 Lightning is licensed under OpenMDW-1.1, which permits commercial use with no meaningful conditions.
Can I run Nemotron 3.5 Lightning on Apple Silicon?
It can run on Apple Silicon through general runtimes, but it is not specifically optimised for it.
What is Nemotron 3.5 Lightning's context window?
Nemotron 3.5 Lightning has a context window of 1,000,000 tokens, about 1000k.

Availability

Recommended for

  • Fast, high-volume agentic workloads on a 24GB card
  • Long-context work where throughput matters
  • Fine-tuning or research needing open training data and recipes

Related models

Run it with

Related guides

Glossary

Our coverage

Catalogue entry last verified 19 August 2026. Specifications change; verify anything you are about to spend money on.