AI glossary
LoRA (low-rank adaptation)
LoRA is a way to fine-tune a model by training small extra matrices beside its frozen weights, which needs far less memory than retraining everything.
Instead of changing a huge weight matrix, LoRA learns two thin matrices whose product is added to it. Only those are trained, often well under one per cent of the parameters.
The result is a small adapter file that can be swapped in and out, so one base model can serve several tasks.
An adapter belongs to its base model. Apply it to a different base, even a newer version of the same family, and it will not work as trained.
Learn it hands-on
These free lessons teach it with commands to run in a lab and checked tasks. Read them whole with no account.