Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning
Fine-tuning pre-trained generative language models to down-stream language generation tasks has shown promising results. However, this comes with the cost of having a single, large model for each task, which is not ideal in low-memory/power scenarios (e.g., mobile). In this paper, we propose an effective way to fine-tune multiple down-stream generation tasks simultaneously using a single, large pre-trained model. The experiments on five diverse language generation tasks show that by just using an additional 2-3% parameters for each task, our model can maintain or even improve the performance of fine-tuning the whole model.
Code (1)
Tasks
Language ModelingLanguage ModellingText GenerationTransfer LearningSimilar Papers 제목 키워드 기반
Exploring the Transferability of Visual Prompting for Multimodal Large Language Models
Although Multimodal Large Language Models (MLLMs) have demonstrated promising versatile capabilities, their performance is still inferior to specialized models on downstream tasks, which makes adaptation necessary to enh…
HallucinationMultimodal ReasoningObject RecognitionVisual PromptingPaliGemma: A versatile 3B VLM for transfer
PaliGemma is an open Vision-Language Model (VLM) that is based on the SigLIP-So400m vision encoder and the Gemma-2B language model. It is trained to be a versatile and broadly knowledgeable base model that is effective t…
Language ModelingLanguage ModellingGenerative flow-based warm start of the variational quantum eigensolver
Hybrid quantum-classical algorithms like the variational quantum eigensolver (VQE) show promise for quantum simulations on near-term quantum devices, but are often limited by complex objective functions and expensive opt…
Robust and Versatile Bipedal Jumping Control through Reinforcement Learning
This work aims to push the limits of agility for bipedal robots by enabling a torque-controlled bipedal robot to perform robust and versatile dynamic jumps in the real world. We present a reinforcement learning framework…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Differentially Private Meta-Learning
Parameter-transfer is a well-known and versatile approach for meta-learning, with applications including few-shot learning, federated learning, and reinforcement learning. However, parameter-transfer algorithms often req…
Federated LearningFew-Shot LearningGeneral Classificationimage-classification+6