LoRA and QLoRA: fine-tuning on a budget

What you’ll learn

What LoRA and QLoRA do, when fine-tuning is genuinely the right approach and when it is not, and which tools make it practical on a single GPU.

When fine-tuning is the right tool

Before any of the how, the most important question is whether to fine-tune at all, because it is often the wrong tool.

Fine-tuning is good at teaching a model a behaviour: a consistent style or tone, a specific output format such as always returning a particular JSON shape, or a way of handling a task it already half-understands. If you keep writing the same long instructions in every prompt, that is a sign fine-tuning could bake them in.

Fine-tuning is a poor and expensive way to teach a model facts. If you want a model to know your documents, RAG is almost always the better answer: cheaper, updatable, and it keeps your data separate from the model. A common and costly mistake is to fine-tune when you should have retrieved.

The concept

A full fine-tune updates every weight in a model, which needs the kind of memory only servers have. LoRA, short for Low-Rank Adaptation, avoids that. It freezes the original weights and trains a small set of extra weights alongside them. Those extra weights, the adapter, are tiny and can be shared or swapped without touching the base model.

QLoRA goes further by applying LoRA on top of a quantised base model, which lowers the memory again. Together they bring fine-tuning of a capable model within reach of a single consumer GPU.

Doing it

The workflow is broadly the same whichever tool you use:

1. Prepare data. Assemble a set of high-quality examples in the format you want the model to learn. Quality and consistency matter far more than quantity.
2. Pick a base model. Start small, such as Qwen3 8B or Llama 3.2 3B. A smaller model that you can iterate on quickly beats a large one you can barely fit.
3. Train a LoRA or QLoRA adapter with a fine-tuning tool.
4. Evaluate honestly on examples the model has not seen, and iterate.

For the tools: Unsloth is the fastest and lightest for a single GPU; Axolotl is config-driven and suits repeatable, production-shaped runs; LLaMA-Factory offers a friendly web UI and broad model support for a first attempt.

What can go wrong

• Fine-tuning to add knowledge. Use RAG for facts. This is the single most common mistake, and it wastes time and money.
• Overfitting. Too many epochs on too little data, and the model parrots your examples and gets worse at everything else. Watch performance on held-out data.
• Poor or inconsistent data. A fine-tune is only as good as its examples. Messy or contradictory data produces a messy model.
• Skipping evaluation. If you do not test on unseen examples, you have no idea whether it worked. It is easy to fool yourself.

Next steps

If your goal is really to give a model knowledge of your own material, start with building a local RAG system instead. If a behaviour is genuinely what you need, compare the fine-tuning tools and start small.