Template-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words. However, when applied to token-level labeling tasks such as NER, it would be time-consuming to enumerate the template queries over all potential entity spans. In this work, we propose a more elegant method to reformulate NER tasks as LM problems without any templates. Specifically, we discard the template construction process while maintaining the word prediction paradigm of pre-training models to predict a class-related pivot word (or label word) at the entity position. Meanwhile, we also explore principled ways to automatically search for appropriate label words that the pre-trained models can easily adapt to. While avoiding complicated template-based process, the proposed LM objective also reduces the gap between different objectives used in pre-training and fine-tuning, thus it can better benefit the few-shot performance. Experimental results demonstrate the effectiveness of the proposed method over bert-tagger and template-based method under few-shot setting. Moreover, the decoding speed of the proposed method is up to 1930.12 times faster than the template-based method.
Code (1)
Tasks
Few-Shot LearningFew-shot NERNamed Entity Recognition (NER)SentenceSimilar Papers 제목 키워드 기반
STT: Soft Template Tuning for Few-Shot Learning
With the rapid expansion of large pre-trained language models, fine-tuning all the model parameters for downstream tasks is becoming computationally prohibitive. The recently developed prompt-based methods freeze the ent…
Few-Shot LearningLanguage ModelingLanguage ModellingMasked Language Modeling+2Template-free Prompt Tuning for Few-shot NER
Prompt-based methods have been successfully applied in sentence-level few-shot learning tasks, mostly owing to the sophisticated design of templates and label words. However, when applied to token-level labeling tasks su…
Few-Shot LearningFew-shot NERNamed Entity Recognition (NER)SentenceSTT: Soft Template Tuning for Few-Shot Adaptation
Prompt tuning has been an extremely effective tool to adapt a pre-trained model to downstream tasks. However, standard prompt-based methods mainly consider the case of sufficient data of downstream tasks. It is still unc…
Few-Shot LearningLanguage ModelingLanguage ModellingMasked Language Modeling+2Zero-Shot Text Classification via Self-Supervised Tuning
Existing solutions to zero-shot text classification either conduct prompting with pre-trained language models, which is sensitive to the choices of templates, or rely on large-scale annotated data of relevant tasks for m…
ClassificationSelf-Supervised LearningSentenceSentiment Analysis+4AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models
Classifiers built upon vision-language models such as CLIP have shown remarkable zero-shot performance across a broad range of image classification tasks. Prior work has studied different ways of automatically creating d…
image-classificationImage ClassificationLanguage ModellingLarge Language Model