Zero-Shot Text Classification with Self-Training
Recent advances in large pretrained language models have increased attention to zero-shot text classification. In particular, models finetuned on natural language inference datasets have been widely adopted as zero-shot classifiers due to their promising results and off-the-shelf availability. However, the fact that such models are unfamiliar with the target task can lead to instability and performance issues. We propose a plug-and-play method to bridge this gap using a simple self-training approach, requiring only the class names along with an unlabeled dataset, and without the need for domain expertise or trial and error. We show that fine-tuning the zero-shot classifier on its most confident predictions leads to significant performance gains across a wide range of text classification tasks, presumably since self-training adapts the zero-shot model to the task at hand.
Code (1)
Tasks
ClassificationNatural Language Inferencetext-classificationText ClassificationZero-Shot Text ClassificationSimilar Papers 제목 키워드 기반
PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification. We formulate text classification as a neural text matching problem where each documen…
ClassificationContrastive Learningtext-classificationText Classification+2Zero-shot Text Classification via Reinforced Self-training
Zero-shot learning has been a tough problem since no labeled data is available for unseen classes during training, especially for classes with low similarity. In this situation, transferring from seen classes to unseen c…
ClassificationGeneral Classificationreinforcement-learningReinforcement Learning (RL)+4Generalized zero-shot audio-to-intent classification
Spoken language understanding systems using audio-only data are gaining popularity, yet their ability to handle unseen intents remains limited. In this study, we propose a generalized zero-shot audio-to-intent classifica…
ClassificationGoal-Oriented Dialogintent-classificationIntent Classification+4WC-SBERT: Zero-Shot Text Classification via SBERT with Self-Training for Wikipedia Categories
Our research focuses on solving the zero-shot text classification problem in NLP, with a particular emphasis on innovative self-training strategies. To achieve this objective, we propose a novel self-training strategy th…
text-classificationText ClassificationZero-Shot Text ClassificationSIGN: Spatial-information Incorporated Generative Network for Generalized Zero-shot Semantic Segmentation
Unlike conventional zero-shot classification, zero-shot semantic segmentation predicts a class label at the pixel level instead of the image level. When solving zero-shot semantic segmentation problems, the need for pixe…
Knowledge DistillationSegmentationSemantic Segmentationzero-shot-classification+2