Run AI on your own hardware.

the tools people arrive for

Answer the questions you arrive with

• 27 October 2026

  MiniMax releases M3.1 Flash, a 428B open-weight MoE under a conditional commercial licence

  MiniMax has published the weights for M3.1 Flash, a multimodal mixture-of-experts model of roughly 428 billion total parameters (about 23 billion active per token) with a context window up to one million tokens, downloadable now in 8-bit. It is open weights rather than open source: the MiniMax Community Licence allows commercial use, but asks for attribution and a one-time notice, and requires separate written authorisation above 20 million US dollars in annual revenue.

• 27 October 2026

  Ollama can now generate speech on macOS, and adds the d3 decision models

  Ollama's 0.40.11 release adds local text-to-speech on macOS: ollama run can read a chat model's reply aloud as the audio streams, and apps can call an OpenAI-compatible speech endpoint. It also adds the d3 family of small decision models in 2B, 4B, 9B and 27B sizes, which classify, answer yes or no, or score options. The speech model and the d3 models arrive with provenance and licence details Ollama has not yet documented, so treat those as unconfirmed.

• 23 October 2026

  China is drafting export controls for frontier AI models, not the open weights you already run

  China's commerce ministry has published a draft that would require an export licence for the technology behind the very largest AI models, and ban it outright for models trained on sensitive state data. The rules are open for comment until 15 November and are not yet law, though a pre-export registration step took effect immediately. For anyone running Chinese open-weight models locally, the practical news is reassuring: weights already released as open source, and access over an API or on a device, are explicitly outside the scope.

• 23 October 2026

  A games studio built Raon-1 mini, an open-weight MoE that targets a single RTX 5090

  KRAFTON, the Korean studio behind PUBG, has released a preview of Raon-1 mini, a 30-billion-parameter mixture-of-experts model it trained from scratch and put out under Apache 2.0 behind a sign-in gate. A 4-bit build is designed to run on one RTX 5090, and it ships with a draft model for speculative decoding. The self-reported benchmarks are strong on maths and reasoning but trail peers on agentic and tool-use work, and it is a preview, so the numbers are the publisher's own.

• Gemma 3 27B

  Google · 27B

  Google's 27-billion-parameter open model, and the largest of the Gemma 3 family. It is multimodal, handling images as well as text, supports over 140 languages, and is designed to run on a single 24GB card at 4-bit.

  Permitted with conditions Text + Image → Text

• Gemma 4 26B-A4B

  Google · MoE 26B (A3.8B)

  Google's efficient Gemma 4 mixture-of-experts: about 26 billion total parameters with only 3.8 billion active per token, so it runs well on modest hardware while holding a 256k context. It reads text and images, and in a notable change for the Gemma line it ships under Apache 2.0.

  Commercial use permitted Text + Image → Text

• Llama 3.3 70B Instruct

  Meta · 70B

  Meta's 70-billion-parameter instruction-tuned model, delivering performance close to their much larger 405B model at a fraction of the hardware cost. A strong general-purpose choice if you have the VRAM for it.

  Permitted with conditions

• Muse Glimmer 30B

  Meta · 30B

  Meta's 30-billion-parameter open model, distilled from its larger closed Muse Spark and built for agentic work: tool use, multi-step reasoning, and recovering from failures. It accepts images as well as text, runs on a single 24GB card at 4-bit, and, unusually for Meta, ships under Apache 2.0 rather than the Llama licence.

  Commercial use permitted Text + Image → Text

• Qwen2.5-Coder 7B Instruct

  Alibaba · 7B

  Alibaba's code-specialised 7B model, strong at code completion and generation well beyond what its size would suggest. A practical choice for a local coding assistant on a mid-range GPU, and permissively licensed.

  Commercial use permitted

• Qwen3 30B-A3B

  Alibaba · MoE 30B-A3B

  A mixture-of-experts model with 30 billion total parameters but only 3 billion active per token, so it generates far faster than a dense 30B while keeping much of the quality. Apache 2.0, with an optional thinking mode.

  Commercial use permitted

• Ollama The fastest route to a working local model. Ollama wraps llama.cpp with simple model management, a one-line pull-and-run workflow, and a local API that many other tools speak to. The usual first recommendation for newcomers.
• LM Studio A polished desktop app for discovering, downloading, and chatting with local models through a graphical interface. A good fit if you would rather not touch a command line, with a built-in local server for connecting other tools.
• llama.cpp The inference engine that much of the local AI ecosystem is built on. A C/C++ implementation that runs GGUF models efficiently across CPU, CUDA, Metal, Vulkan, and more, with fine control over how inference behaves.
• vLLM A high-throughput inference and serving engine built to maximise GPU utilisation and handle many concurrent requests. Its PagedAttention and continuous batching make it the common choice once you are serving a model to more than one user.
• ComfyUI A node-graph interface for local image generation, where you build a workflow by wiring together nodes. It is the most flexible and powerful local image tool, quick to support new models, and it saves workflows as reusable JSON.
• Continue An open-source IDE extension for VS Code and JetBrains that adds AI autocomplete, chat, and multi-file edits, with first-class support for local models through Ollama. A common starting point for a fully local coding setup.
• LangChain The most widely used framework for building LLM applications, with a huge set of integrations for models, vector stores, and tools. It is a common backbone for RAG systems, and works with local models through Ollama and llama.cpp.
• Qdrant A fast, production-grade vector database written in Rust, with strong metadata filtering and a clean operational story. A common choice once a RAG system outgrows a prototype store, and free to self-host under Apache 2.0.

• 🚀 Getting Started What local AI is, why it matters, and how to run your first model.
• 🖥️ Hardware GPUs, unified memory, VRAM, and what you actually need for what you want to do.
• ⚙️ Running Models Quantisation, inference engines, model formats, and getting good performance.
• 🧪 Evaluation & Benchmarking How to test, benchmark and compare local models: reading public benchmarks honestly, and designing your own tests.
• 💻 Local AI Coding Code assistants, autocomplete, and agentic coding tools that run on your machine.
• 📚 RAG & Knowledge Systems Building retrieval systems over your own documents, entirely locally.
• 🤖 Agentic AI & Harnesses Tool use, agent frameworks, and orchestration with local models.
• 🎯 Fine-Tuning Adapting models to your domain: LoRA, QLoRA, full fine-tunes, and when each is worth it.
• 🎨 Beyond LLMs Image generation, speech recognition, text-to-speech, and other local models.
• 🏢 Server & Enterprise Self-hosting at organisational scale, from a small team on a couple of GPUs to enterprise clusters running open models beyond 100 billion parameters.
• 🛡️ AI Regulation & Sovereignty AI-specific regulation like the EU AI Act, alongside data residency, GDPR, and the practical case for keeping data in-house.

• Agent
• Chunking
• Context window
• ControlNet
• Copyleft (GPL / AGPL)
• Diffusion
• Embedding
• Fine-tuning
• GGUF
• Inference engine
• Instruction tuning
• KV cache