STT: Soft Template Tuning for Few-Shot Learning
With the rapid expansion of large pre-trained language models, fine-tuning all the model parameters for downstream tasks is becoming computationally prohibitive. The recently developed prompt-based methods freeze the entire model parameters and only update the so-called prompt parameters appended to the inputs, significantly reducing the burden of fully fine-tuning. However, standard prompt-based methods mainly consider the case where sufficient data of downstream tasks are available. It is still unclear whether the advantage can be transferred to the few-shot regime, where only limited data are available for each downstream task. Our empirical studies suggest there is still a gap between prompt tuning and fully fine-tuning for few-shot learning. We propose a new prompt-tuning framework, called Soft Template Tuning (STT), to bridge the gap. STT combines manual prompts and auto-prompts, and treats downstream classification tasks as a masked language modeling task. STT can close the gap between fine-tuning and prompt-based methods without introducing additional parameters. Importantly, it can even outperform the time- and resource-consuming fine-tuning method on sentiment classification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Few-Shot LearningLanguage ModelingLanguage ModellingMasked Language ModelingSentiment AnalysisSentiment ClassificationSimilar Papers 제목 키워드 기반
STT: Soft Template Tuning for Few-Shot Adaptation
Prompt tuning has been an extremely effective tool to adapt a pre-trained model to downstream tasks. However, standard prompt-based methods mainly consider the case of sufficient data of downstream tasks. It is still unc…
Few-Shot LearningLanguage ModelingLanguage ModellingMasked Language Modeling+2BvSP: Broad-view Soft Prompting for Few-Shot Aspect Sentiment Quad Prediction
Aspect sentiment quad prediction (ASQP) aims to predict four aspect-based elements, including aspect term, opinion term, aspect category, and sentiment polarity. In practice, unseen aspects, due to distinct data distribu…
Language ModelingLanguage ModellingSentenceICL Markup: Structuring In-Context Learning using Soft-Token Tags
Large pretrained language models (LLMs) can be rapidly adapted to a wide variety of tasks via a text-to-text approach, where the instruction and input are fed to the model in natural language. Combined with in-context le…
In-Context LearningIntent DetectionMeta-Learningparameter-efficient fine-tuning+2Template-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words. However, when applied to token-level labeling tasks su…
Few-Shot LearningFew-shot NERNamed Entity Recognition (NER)SentenceTemplate-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words. However, when applied to token-level labeling tasks su…
Few-Shot LearningFew-shot NERNamed Entity Recognition (NER)Sentence