Skip to content
local-ai

Running Models

Quantisation, inference engines, model formats, and getting good performance.

  1. Intermediate

    Understanding quantisation

    Quantisation is what makes local AI possible on consumer hardware. What the formats mean, what you give up, and how to choose one for your memory budget.