QLoRA

QLoRA squeezes fine-tuning onto hardware you might actually own. The trick is to store the frozen base model in a heavily compressed, quantized form — roughly four bits per weight instead of sixteen — so a model that would not fit in your GPU's memory suddenly does, and then to train ordinary LoRA adapters on top of it.

The base weights are quantized to four bits and kept frozen; gradients pass through them only to update the small LoRA matrices, which stay in higher precision. Careful details — a data type matched to how weights are distributed, and paging optimizer state out to CPU memory when usage spikes — keep quality close to full-precision fine-tuning. The headline result was fine-tuning a 65-billion-parameter model on a single consumer-grade GPU.

QLoRA democratised fine-tuning: hobbyists and small labs could suddenly adapt large open models. The trade-offs are slightly slower training and the usual small quality cost that comes with quantization.