Inference engine
Also: inference server, runtime
The software that actually runs a model to produce output, handling the model format, the hardware, quantisation, and memory. llama.cpp, vLLM, and MLX are examples.
A set of model weights on disk does nothing on its own; an inference engine is the program that loads them and runs them to generate output. It deals with the model format, the quantisation, the hardware (GPU, CPU, or Apple Silicon), and how memory is used.
Different engines suit different goals. llama.cpp and the apps built on it target single-user local use across all hardware; vLLM and SGLang target high-throughput serving to many users; MLX targets Apple Silicon. Choosing the right one is often as important as choosing the model. See inference engines explained.