Exploiting Cloze-Questions for Few-Shot Text Classification and Natural Language Inference
Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with {``}task descriptions{''} in natural language (e.g., Radford et al., 2019). While this approach underperforms its supervised counterpart, we show in this work that the two ideas can be combined: We introduce Pattern-Exploiting Training (PET), a semi-supervised training procedure that reformulates input examples as cloze-style phrases to help language models understand a given task. These phrases are then used to assign soft labels to a large set of unlabeled examples. Finally, standard supervised training is performed on the resulting training set. For several tasks and languages, PET outperforms supervised training and strong semi-supervised approaches in low-resource settings by a large margin.
Code (1)
Tasks
Few-Shot Text ClassificationLanguage ModelingLanguage ModellingNatural Language Inferencetext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference
Some NLP tasks can be solved in a fully unsupervised fashion by providing a pretrained language model with "task descriptions" in natural language (e.g., Radford et al., 2019). While this approach underperforms its super…
Few-Shot Text ClassificationGeneral ClassificationLanguage ModelingLanguage Modelling+2Few-shot Named Entity Recognition with Cloze Questions
Despite the huge and continuous advances in computational linguistics, the lack of annotated data for Named Entity Recognition (NER) is still a challenging issue, especially in low-resource languages and when domain know…
Few-Shot Learningnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+1Exploiting Portuguese Lexical Knowledge Bases for Answering Open Domain Cloze Questions Automatically
We present the task of answering cloze questions automatically and how it can be tackled by exploiting lexical knowledge bases (LKBs). This task was performed in what can be seen as an indirect evaluation of Portuguese L…
Question AnsweringZero-shot Commonsense Question Answering with Cloze Translation and Consistency Optimization
Commonsense question answering (CQA) aims to test if models can answer questions regarding commonsense knowledge that everyone knows. Prior works that incorporate external knowledge bases have shown promising results, bu…
Natural QuestionsQuestion AnsweringRelation Extractiontext-classification+2Automatically Identifying Words That Can Serve as Labels for Few-Shot Text Classification
A recent approach for few-shot text classification is to convert textual inputs to cloze questions that contain some form of task description, process them with a pretrained language model and map the predicted words to …
Few-Shot Text ClassificationGeneral ClassificationLanguage ModelingLanguage Modelling+2