Solutions catalogue
The tools that make local AI work: inference engines, desktop apps, coding assistants, RAG frameworks, and more. Every entry carries an honest weaknesses list, whatever its commercial relationship with us.
Browse by category
55 solutions.
- Coding toolsOpen source
A terminal-native AI pair programmer that works directly with your git repository, making changes as commits you can review and revert. Model agnostic, and works with local models through Ollama or any OpenAI-compatible endpoint.
macOSWindowsLinux - Desktop appsOpen source
An all-in-one local AI application from Mintplex Labs, built around document chat and agents. It bundles an LLM connector, an embedder, and a LanceDB vector store, so you can index and chat with your own documents out of the box.
macOSWindowsLinuxDocker - MonitoringProprietary
A self-hostable LLM observability tool for tracing, evaluating, and debugging AI applications, built on OpenTelemetry.
macOSWindowsLinuxDockerWeb - Agent frameworksOpen source
A Microsoft programming framework for building multi-agent AI applications, with layered Core, AgentChat, and Extensions APIs.
macOSWindowsLinux - Fine-tuningOpen source
A config-driven fine-tuning framework where you describe a training run in a YAML file. It supports the full range of techniques, from LoRA and QLoRA to full fine-tunes and preference optimisation, and scales to multi-GPU setups.
Linux - Voice & audioOpen source
An open-source text-to-speech model and tool from Resemble AI, notable for zero-shot voice cloning from a short sample and control over emotional expression. MIT licensed, so it is one of the few cloning-capable options you can use commercially.
macOSWindowsLinux - Vector databasesOpen source
An open-source vector database designed to be easy to start with, and a common default for local RAG prototypes. It runs embedded in your application or as a server, and needs almost no setup to get going.
macOSWindowsLinuxDocker - Coding toolsOpen source
An open-source coding agent for VS Code and JetBrains that plans changes, then acts with permissioned access to files, the terminal, the browser, and MCP tools. Works with local models, so an agentic workflow can stay on your machine.
macOSWindowsLinux - Image generationOpen source
A node-graph interface for local image generation, where you build a workflow by wiring together nodes. It is the most flexible and powerful local image tool, quick to support new models, and it saves workflows as reusable JSON.
macOSWindowsLinux - Coding toolsOpen source
An open-source IDE extension for VS Code and JetBrains that adds AI autocomplete, chat, and multi-file edits, with first-class support for local models through Ollama. A common starting point for a fully local coding setup.
macOSWindowsLinux - Agent frameworksOpen source
A framework for building multi-agent systems around an intuitive role-playing paradigm: you define agents with roles and goals, and assemble them into a crew. It is one of the easiest agent frameworks to get working with local models.
macOSWindowsLinux - Agent frameworksOpen source
An open-source platform for visually building agentic AI workflows, RAG pipelines, and LLM apps, with multi-model support.
DockerWeb - Inference enginesOpen source
A fast inference library for running GPTQ and EXL2 quantised models on consumer NVIDIA GPUs, known for high throughput at low bit-widths.
WindowsLinux - Voice & audioOpen source
A faster reimplementation of OpenAI Whisper using the CTranslate2 engine, the common way to run Whisper efficiently for local transcription.
macOSWindowsLinuxDocker - Image generationOpen source
A deliberately simple, SDXL-focused image generation app that hides most of the usual settings so you can concentrate on prompting. It is one of the gentlest ways into local image generation, though it is now in maintenance only.
macOSWindowsLinux - Desktop appsOpen source
A desktop app from Nomic focused on simple, fully offline chat, including chat over your own documents. It runs on ordinary hardware without a GPU, and is one of the least fiddly ways to get private local AI working.
macOSWindowsLinux - RAG frameworksOpen source
An enterprise-focused framework from deepset for building production RAG and search systems. Its modular pipeline architecture and evaluation tools make it a solid choice for serious, maintainable deployments, and it runs with local models.
macOSWindowsLinux - Image generationOpen source
A local image generation app built around an artist-facing unified canvas, with inpainting, outpainting, a gallery, and model management in one place. It also offers node workflows, but leads with a production-friendly editing experience.
macOSWindowsLinux - Desktop appsOpen source
An open-source, offline-first desktop app for chatting with local models through a clean interface. It runs models locally by default and keeps your conversations on your machine, with a focus on privacy.
macOSWindowsLinux - Inference enginesOpen source
A self-contained, single-executable inference tool built on llama.cpp, running GGUF models with a built-in web UI plus image, voice, and vision features.
macOSWindowsLinuxDocker - Vector databasesOpen source
An embedded, serverless vector database that stores data in the columnar Lance format and can sit directly on local disk or object storage. It suits file-based, in-process use where you do not want to run a separate database server.
macOSWindowsLinux - RAG frameworksOpen source
The most widely used framework for building LLM applications, with a huge set of integrations for models, vector stores, and tools. It is a common backbone for RAG systems, and works with local models through Ollama and llama.cpp.
macOSWindowsLinux - MonitoringOpen source
An open-source observability platform for LLM applications: tracing, cost and latency tracking, evaluations, datasets, and prompt management. It is self-hostable, so you can keep your traces on your own infrastructure.
DockerWebLinux - Agent frameworksOpen source
A graph-based framework from the LangChain team for building stateful, controllable agent workflows. You define agents as graphs with explicit state, branching, and human-in-the-loop steps, which suits production systems that need audit trails and recovery points. Works with local models.
macOSWindowsLinux - Agent frameworksOpen source
An open-source framework for stateful agents with long-term memory, formerly known as MemGPT. It gives agents transparent, persistent memory so they can remember facts across sessions, which general-purpose frameworks do not ship by default.
macOSWindowsLinuxDocker - OrchestrationOpen source
A gateway and SDK that gives you one OpenAI-compatible API across hundreds of model providers, local backends included. As a proxy it adds routing, rate limiting, cost tracking, and key management in front of your models.
DockerLinuxmacOSWindows - Fine-tuningOpen source
A fine-tuning toolkit known for broad model support and an approachable web UI, which makes it one of the friendliest ways to start fine-tuning. It covers LoRA, QLoRA, full fine-tunes, and preference optimisation across many model families.
LinuxWindows - OrchestrationOpen source
A lightweight Go proxy that hot-swaps models behind an OpenAI-compatible endpoint, commonly used in front of llama.cpp to serve many models from one machine.
macOSWindowsLinuxDocker - Inference enginesOpen source
The inference engine that much of the local AI ecosystem is built on. A C/C++ implementation that runs GGUF models efficiently across CPU, CUDA, Metal, Vulkan, and more, with fine control over how inference behaves.
macOSWindowsLinux - RAG frameworksOpen source
A data framework focused on connecting LLMs to your own data, with strong document loading, indexing, and retrieval. It is a natural choice when the heart of your application is RAG over documents, and works with local models.
macOSWindowsLinux - Desktop appsProprietary
A polished desktop app for discovering, downloading, and chatting with local models through a graphical interface. A good fit if you would rather not touch a command line, with a built-in local server for connecting other tools.
macOSWindowsLinux - OrchestrationOpen source
A self-hosted, OpenAI and Anthropic-compatible API server that runs LLM, vision, audio, image, and video models locally across many backends.
macOSWindowsLinuxDocker - Vector databasesOpen source
A distributed, cloud-native vector database built for high-performance similarity search over billions of vectors.
LinuxmacOSDocker - Inference enginesOpen source
Apple's open-source array and machine-learning framework, built for Apple Silicon's unified memory. With MLX-LM it runs language models efficiently on Macs, and is often the fastest path to good local performance on Apple hardware.
macOS - Fine-tuningOpen source
A Python package for running and fine-tuning LLMs on Apple Silicon via the MLX framework, the native path for training on a Mac.
macOS - Desktop appsOpen source
The fastest route to a working local model. Ollama wraps llama.cpp with simple model management, a one-line pull-and-run workflow, and a local API that many other tools speak to. The usual first recommendation for newcomers.
macOSWindowsLinuxDocker - Desktop appsOpen source
A self-hosted web interface for local models, most commonly paired with Ollama. It adds multi-user accounts, document chat and RAG, tools, and shared workspaces on top of a model backend, all running on your own server.
DockerWebLinuxmacOSWindows - Coding toolsOpen source
An autonomous coding agent, formerly OpenDevin, built to take on feature-sized tasks end to end. It runs each task in an isolated Docker sandbox, and works with local models for a fully self-hosted agent.
LinuxmacOSWindowsDocker - Vector databasesOpen source
An open-source extension that adds vector similarity search to PostgreSQL, so you can store embeddings alongside your existing relational data with no extra service. The simplest path if you already run Postgres.
macOSWindowsLinuxDocker - Voice & audioOpen source
A fast, lightweight text-to-speech engine designed to run offline on low-resource hardware, right down to a Raspberry Pi. It is the default local voice for Home Assistant, with many voices across a range of languages.
LinuxmacOSWindows - Agent frameworksOpen source
A Python agent framework from the Pydantic team, built around type safety and structured, validated outputs. It is model-agnostic, works with local models, and brings a clean, familiar developer experience to building agents.
macOSWindowsLinux - Vector databasesOpen source
A fast, production-grade vector database written in Rust, with strong metadata filtering and a clean operational story. A common choice once a RAG system outgrows a prototype store, and free to self-host under Apache 2.0.
LinuxDockermacOSWindows - RAG frameworksOpen source
An open-source RAG engine built around deep document understanding, template-based chunking, and grounded, citation-backed answers.
LinuxDocker - Inference enginesOpen source
A high-performance serving engine notable for fast structured output and efficient KV-cache sharing. Its RadixAttention makes it particularly strong for prefix-heavy RAG and multi-turn chat, where reused context is the main lever.
LinuxDocker - Vector databasesOpen source
A tiny vector search extension for SQLite, written in C with no dependencies, so it runs anywhere SQLite does, from a server to a phone. A neat fit when you want vector search inside an existing SQLite database with no extra service.
macOSWindowsLinux - Image generationOpen source
The widely adopted Gradio-based web interface for Stable Diffusion, offering text-to-image, image-to-image, and extensive editing tools.
macOSWindowsLinux - Image generationOpen source
A performance- and memory-optimised platform built on the AUTOMATIC1111 web UI, aimed at faster inference and newer models like FLUX.
macOSWindowsLinux - Inference enginesOpen source
NVIDIA's library for extracting peak inference performance from NVIDIA GPUs. It compiles optimised engines for a model and hardware pairing, delivering the best throughput and latency available on supported cards, at the cost of setup complexity.
LinuxDocker - RAG frameworksOpen source
Hugging Face's fast serving solution for text-embedding, reranking, and sequence-classification models, the retrieval-side counterpart to a text-generation server.
macOSLinuxDocker - OrchestrationOpen source
Hugging Face's production-grade Rust and Python server for serving large language models with optimised, high-throughput inference.
LinuxDocker - Fine-tuningOpen source
A fine-tuning library focused on speed and low memory use, with optimised kernels that make LoRA and QLoRA training much faster and lighter. It is the tool to reach for when you want to fine-tune on a single consumer GPU.
LinuxWindows - Inference enginesOpen source
A high-throughput inference and serving engine built to maximise GPU utilisation and handle many concurrent requests. Its PagedAttention and continuous batching make it the common choice once you are serving a model to more than one user.
LinuxDocker - Vector databasesOpen source
An open-source, cloud-native vector database combining semantic search, keyword filtering, RAG, and reranking in one query interface.
LinuxDocker - Voice & audioOpen source
An efficient C and C++ implementation of OpenAI's Whisper for local transcription, from the same lineage as llama.cpp. It runs Whisper well across CPU, NVIDIA, and Apple Silicon, which makes local speech-to-text practical on ordinary hardware.
macOSWindowsLinux - Coding toolsOpen source
A fast, GPU-accelerated code editor written in Rust, with open-source AI features built in. It can use local models through Ollama or any OpenAI-compatible endpoint, making it a native alternative to AI-first editors.
macOSLinuxWindows