Improving Instruct Models for Free: A Study on Partial Adaptation
Instruct models, obtained from various instruction tuning or post-training steps, are commonly deemed superior and more usable than their base counterpart. While the model gains instruction following ability, instruction tuning may lead to forgetting the knowledge from pre-training or it may encourage the model being overly conversational or verbose. This, in turn, can lead to degradation of in-context few-shot learning performance. In this work, we study the performance trajectory between base and instruct models by scaling down the strength of instruction-tuning via the partial adaption method. We show that, across several model families and model sizes, reducing the strength of instruction-tuning results in material improvement on a few-shot in-context learning benchmark covering a variety of classic natural language tasks. This comes at the cost of losing some degree of instruction following ability as measured by AlpacaEval. Our study shines light on the potential trade-off between in-context learning and instruction following abilities that is worth considering in practice.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningIn-Context LearningInstruction FollowingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SAO-Instruct: Free-form Audio Editing using Natural Language Instructions
Generative models have made significant progress in synthesizing high-fidelity audio from short textual descriptions. However, editing existing audio using natural language has remained largely underexplored. Current app…
Confidence Score Weighting Adaptation for Source-Free Unsupervised Domain Adaptation
Unsupervised domain adaptation (UDA) aims to achieve high performance within the unlabeled target domain by leveraging the labeled source domain. Source-free UDA, which is a more challenging UDA task, can access the pre-…
Domain AdaptationPseudo LabelUnsupervised Domain AdaptationOneRing: A Simple Method for Source-free Open-partial Domain Adaptation
In this paper, we investigate Source-free Open-partial Domain Adaptation (SF-OPDA), which addresses the situation where there exist both domain and category shifts between source and target domains. Under the SF-OPDA set…
Domain AdaptationDomain GeneralizationOpen Set LearningPartial Domain Adaptation+1JMedLoRA:Medical Domain Adaptation on Japanese Large Language Models using Instruction-tuning
In the ongoing wave of impact driven by large language models (LLMs) like ChatGPT, the adaptation of LLMs to medical domain has emerged as a crucial research frontier. Since mainstream LLMs tend to be designed for genera…
Domain AdaptationMedical Question AnsweringMultiple-choiceQuestion AnsweringPRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA
With the rapid scaling of large language models (LLMs), serving numerous low-rank adaptations (LoRAs) concurrently has become increasingly impractical, leading to unaffordable costs and necessitating more parameter-effic…