ZeroBERTo: Leveraging Zero-Shot Text Classification by Topic Modeling
Traditional text classification approaches often require a good amount of labeled data, which is difficult to obtain, especially in restricted domains or less widespread languages. This lack of labeled data has led to the rise of low-resource methods, that assume low data availability in natural language processing. Among them, zero-shot learning stands out, which consists of learning a classifier without any previously labeled data. The best results reported with this approach use language models such as Transformers, but fall into two problems: high execution time and inability to handle long texts as input. This paper proposes a new model, ZeroBERTo, which leverages an unsupervised clustering step to obtain a compressed data representation before the classification task. We show that ZeroBERTo has better performance for long inputs and shorter execution time, outperforming XLM-R by about 12% in the F1 score in the FolhaUOL dataset. Keywords: Low-Resource NLP, Unlabeled data, Zero-Shot Learning, Topic Modeling, Transformers.
Code (0)
등록된 구현이 없습니다.
Tasks
Classificationtext-classificationText ClassificationXLM-RZero-Shot LearningZero-Shot Text ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Leveraging Type Descriptions for Zero-shot Named Entity Recognition and Classification
A common issue in real-world applications of named entity recognition and classification (NERC) is the absence of annotated data for the target entity classes during training. Zero-shot learning approaches address this i…
Machine Reading Comprehensionnamed-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)+6TSPE: Task-Specific Prompt Ensemble for Improved Zero-Shot Audio Classification
Audio-language models (ALMs) excel in zero-shot audio classification, a task where models classify previously unseen audio clips at test time by leveraging descriptive natural language prompts. We introduce TSPE (Task-Sp…
Audio ClassificationClassificationDescriptiveZero-shot Audio ClassificationA sound description: Exploring prompt templates and class descriptions to enhance zero-shot audio classification
Audio-text models trained via contrastive learning offer a practical approach to perform audio classification through natural language prompts, such as "this is a sound of" followed by category names. In this work, we ex…
Audio ClassificationClassificationContrastive LearningPrompt Engineering+1Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval
Contrastive language-image pre-training (CLIP) has demonstrated remarkable zero-shot classification ability, namely image classification using novel text labels. Existing works have attempted to enhance CLIP by fine-tuni…
Cross-Modal Retrievalimage-classificationImage ClassificationRetrieval+3Improving Cross-lingual Text Classification with Zero-shot Instance-Weighting
Cross-lingual text classification (CLTC) is a challenging task made even harder still due to the lack of labeled data in low-resource languages. In this paper, we propose zero-shot instance-weighting, a general model-agn…
Classificationtext-classificationText ClassificationZero-Shot Learning