The cost of full fine-tuning
Standard fine-tuning adjusts all of a model's weights. For a large model that means holding optimizer state for billions of parameters in memory during training (often several times the model's size), and ending up with a full copy of the model per task. If you want ten specialized variants, that's ten full models to store, load, and serve. Parameter-efficient fine-tuning (PEFT) is the family of methods that avoids this, and LoRA is the one that won.
The key observation: the update is low-rank
When you fine-tune, the change you make to a big weight matrix turns out to be far simpler than the matrix itself — it has low rank, meaning it can be reconstructed from a handful of underlying directions. LoRA (Low-Rank Adaptation) exploits this directly: freeze the original weight matrix entirely, and represent the update as the product of two skinny matrices, A (tall and thin) and B (thin and wide).
A full update to a 4096×4096 matrix is ~16 million numbers. Approximate it with two matrices of rank 8 and you train 4096×8 + 8×4096 ≈ 65 thousand numbers — a tiny fraction. Only A and B are trained; the original weights never move. That is where the “10,000× fewer trainable parameters” figure comes from.
Why it has zero inference overhead
At training time the model computes its normal output plus a small correction from the A·B adapter. The elegant part: because the adapter is just a matrix product added to the frozen weight, you can merge it back into the original weights after training — fold A·B into W to get a single updated matrix. The merged model is bit-for-bit a normal model of the same size and speed. Unlike older adapter methods that inserted extra layers (and extra latency), a merged LoRA adds nothing at inference.
Hot-swappable behaviors
Because each LoRA adapter is small (megabytes, not gigabytes), you can keep many adapters for one base model and load them on demand: a legal-tone adapter, a JSON-format adapter, a customer-specific adapter. Serving systems can even keep the base model resident and swap adapters per request. This is the economic unlock — one expensive base model in memory, many cheap specializations layered on top, instead of a full model per use case.
QLoRA pushed it further by quantizing the frozen base model to 4-bit while training the adapter in higher precision, making it possible to fine-tune very large models on a single consumer GPU. The combination — quantized frozen base, small high-precision adapter — is why fine-tuning large models became accessible outside big labs.
When LoRA is enough — and its limits
For most fine-tuning goals — tone, format, a domain, instruction-following on a narrow task — LoRA matches full fine-tuning closely at a fraction of the cost, and it's the right default. Its limits show up when you're trying to install genuinely newbroad capability or substantially shift the model's knowledge: the low-rank update is, by design, a small nudge, and a small nudge can only do so much. For that you need more data, higher rank, or full fine-tuning — but those cases are rarer than people assume.
The takeaway: LoRA reframed fine-tuning from “retrain and ship a new model” to “train a small, mergeable, swappable patch.” It's why the practical answer to “can we customize this model?” is now usually yes, cheaply.