LoRA Learns Less and Forgets Less
Low-Rank Adaptation (LoRA) is a widely-used parameter-efficient finetuning method for large language models. LoRA saves memory by training only low rank perturbations to selected weight matrices. In this work, we compare the performance of LoRA and full finetuning on two target domains, programming and mathematics. We consider both the instruction finetuning (approximately 100K prompt-response pairs) and continued pretraining (20B unstructured tokens) data regimes. Our results show that, in the standard low-rank settings, LoRA substantially underperforms full finetuning. Nevertheless, LoRA better maintains the base model's performance on tasks outside the target domain. We show that LoRA mitigates forgetting more than common regularization techniques such as weight decay and dropout; it also helps maintain more diverse generations. Finally, we show that full finetuning learns perturbations with a rank that is 10-100X greater than typical LoRA configurations, possibly explaining some of the reported gaps. We conclude by proposing best practices for finetuning with LoRA.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Gender Fairness of Pre-Trained Language Models without Catastrophic Forgetting
Existing studies addressing gender bias of pre-trained language models, usually build a small gender-neutral data set and conduct a second phase pre-training on the model with such data. However, given the limited size a…
coreference-resolutionCoreference ResolutionFairnessOptimizer-Model Consistency: Full Finetuning with the Same Optimizer as Pretraining Forgets Less
Optimizers play an important role in both pretraining and finetuning stages when training large language models (LLMs). In this paper, we present an observation that full finetuning with the same optimizer as in pretrain…
Continual Learning through Human-Robot Interaction: Human Perceptions of a Continual Learning Robot in Repeated Interactions
For long-term deployment in dynamic real-world environments, assistive robots must continue to learn and adapt to their environments. Researchers have developed various computational models for continual learning (CL) th…
Continual LearningObject RecognitionLESSON: Learning to Integrate Exploration Strategies for Reinforcement Learning via an Option Framework
In this paper, a unified framework for exploration in reinforcement learning (RL) is proposed based on an option-critic model. The proposed framework learns to integrate a set of diverse exploration strategies so that th…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)ServerlessLoRA: Minimizing Latency and Cost in Serverless Inference for LoRA-Based LLMs
Serverless computing has grown rapidly for serving Large Language Model (LLM) inference due to its pay-as-you-go pricing, fine-grained GPU usage, and rapid scaling. However, our analysis reveals that current serverless c…
GPULarge Language Model