LoRA: Low-Rank Adaptation of Large Language Models
https://arxiv.org/abs/2106.09685
LoRA is a technique used to fine-tune LLMs in a more efficient and memory-friendly way.
Low-Rank Decomposition: Instead of updating all parameters of a pre-trained model, LoRA introduces a small set of trainable low-rank matrices that adapt the weights of specific layers.