BBTv2: Towards a Gradient-Free Future with Large Language Models
Most downstream adaptation methods tune all or part of the parameters of pre-trained models (PTMs) through gradient descent, where the tuning cost increases linearly with the growth of the model size. By contrast, gradient-free methods only require the forward computation of the PTM to tune the prompt, retaining the benefits of efficient tuning and deployment. Though, past work on gradient-free tuning often introduces gradient descent to seek a good initialization of prompt and lacks versatility across tasks and PTMs. In this paper, we present BBTv2, an improved version of Black-Box Tuning, to drive PTMs for few-shot learning. We prepend continuous prompts to every layer of the PTM and propose a divide-and-conquer gradient-free algorithm to optimize the prompts at different layers alternately. Extensive experiments across various tasks and PTMs show that BBTv2 can achieve comparable performance to full model tuning and state-of-the-art parameter-efficient methods (e.g., Adapter, LoRA, BitFit, etc.) under few-shot settings while maintaining much fewer tunable parameters.
Code (1)
Tasks
Few-Shot LearningLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Technical Report -- Competition Solution for Prompt Tuning using Pretrained Language Model
Prompt tuning recently becomes a hot-spot in the applications of large pretrained language models on specific downstream tasks. Regarding the Language Model as a Service (LMaaS), black-box tuning using derivative-free op…
Few-Shot LearningLanguage ModelingLanguage ModellingGradient Sparsification For \emph{Masked Fine-Tuning} of Transformers
Fine-tuning masked language models is widely adopted for transfer learning to downstream tasks and can be achieved by (1) freezing gradients of the pretrained network or only updating gradients of a newly added classific…
Transfer LearningXLM-RGradient Sparsification For Masked Fine-Tuning of Transformers
Fine-tuning pretrained self-supervised language models is widely adopted for transfer learning to downstream tasks. Fine-tuning can be achieved by freezing gradients of the pretrained network and only updating gradients …
Transfer LearningFuture-KL Regularized GRPO: Process-Level Credit Assignment from $f$-Divergence Regularization
Group Relative Policy Optimization (GRPO) is widely used for critic-free Large Language Model (LLM) post-training, but its KL regularization is usually implemented as a local loss-side token penalty. We show that this mi…
Reinforcement LearningMathematical ReasoningWhen Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario
Large pre-trained language models (PLMs) have garnered significant attention for their versatility and potential for solving a wide spectrum of natural language processing (NLP) tasks. However, the cost of running these …
Knowledge Distillation