Glossary
The vocabulary of local AI, explained plainly. Technical but accessible, and cross-linked throughout the site.
A
C
- Chunking Splitting documents into smaller passages before embedding them for retrieval. How you chunk strongly affects the quality of a RAG system.
- Context window The maximum amount of text, measured in tokens, a model can consider at once, covering both your input and its output.
- ControlNet An add-on for diffusion image models that conditions generation on an input such as a pose, depth map, or edge outline, giving precise control over composition.
- Copyleft (GPL / AGPL) A licence style that requires derivative works to be released under the same terms. It matters for local AI tools: an AGPL tool built into a product you distribute can oblige you to open your own source.
D
E
F
G
I
- Inference engine The software that actually runs a model to produce output, handling the model format, the hardware, quantisation, and memory. llama.cpp, vLLM, and MLX are examples.
- Instruction tuning Fine-tuning a base model to follow instructions and hold a conversation. The "instruct" or "chat" version of a model has had this treatment; the "base" version has not.
K
L
M
- MCP The Model Context Protocol: an open standard for connecting models to external tools and data through a common interface, so an agent can use many tools without a bespoke integration for each.
- Mixture-of-experts A model design where only a fraction of the parameters are used for each token, so a large model can generate at closer to the speed of a much smaller one.
- MLX Apple's machine-learning framework for Apple Silicon, and the model format built on it. MLX builds run models efficiently using a Mac's unified memory.
O
Q
- QLoRA LoRA fine-tuning performed on a quantised base model, so a large model can be adapted on a single consumer GPU. It combines quantisation with low-rank adapters.
- Quantisation Storing a model's weights at lower numerical precision to shrink its memory footprint, trading a little quality for a much smaller size.
R
- RAG Retrieval-augmented generation: giving a model relevant snippets from your own documents at question time, so it can answer from them rather than from memory.
- Reranker A model that re-scores an initial set of retrieved passages by how well each actually answers the query, sharpening RAG results before they reach the main model.
S
T
- Text-to-video Generating a short video clip from a text prompt with a diffusion model. It is far more memory- and compute-intensive than image generation, and open models are only recently consumer-runnable.
- Throughput How fast a model generates text, usually measured in tokens per second. It depends on the hardware, the model size, the quantisation, and how many requests run at once.
- Tokenizer The component that breaks text into tokens, the sub-word units a model actually reads and generates. Token counts, not word counts, drive context limits and API pricing.
- Tool use A model calling external functions, such as search, code execution, or an API, by emitting a structured request that the surrounding software runs. It is the foundation of agents.
U
V
- Vector database A database that stores embeddings and quickly finds the ones closest to a query. It is the retrieval half of a RAG system.
- Voice cloning Text-to-speech that reproduces a specific target voice from a short sample, rather than using a fixed built-in voice. Powerful, and with obvious consent and misuse considerations.
- VRAM The memory on a graphics card. A model must fit in it (alongside its context) to run on the GPU, so VRAM is the single biggest constraint on what you can run.