Weakly Supervised Text Classification using Supervision Signals from a Language Model
Solving text classification in a weakly supervised manner is important for real-world applications where human annotations are scarce. In this paper, we propose to query a masked language model with cloze style prompts to obtain supervision signals. We design a prompt which combines the document itself and "this article is talking about [MASK]." A masked language model can generate words for the [MASK] token. The generated words which summarize the content of a document can be utilized as supervision signals. We propose a latent variable model to learn a word distribution learner which associates generated words to pre-defined categories and a document classifier simultaneously without using any annotated data. Evaluation on three datasets, AGNews, 20Newsgroups, and UCINews, shows that our method can outperform baselines by 2%, 4%, and 3%.
Code (1)
Tasks
Language ModelingLanguage Modellingtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Constrained Labeling for Weakly Supervised Learning
Curation of large fully supervised datasets has become one of the major roadblocks for machine learning. Weak supervision provides an alternative to supervised learning by training with cheap, noisy, and possibly correla…
image-classificationImage ClassificationWeakly-supervised LearningMEGClass: Extremely Weakly Supervised Text Classification via Mutually-Enhancing Text Granularities
Text classification is essential for organizing unstructured text. Traditional methods rely on human annotations or, more recently, a set of class seed words for supervision, which can be costly, particularly for special…
text-classificationText ClassificationWeakly-Supervised Hierarchical Text Classification
Hierarchical text classification, which aims to classify text documents into a given hierarchy, is an important task in many real-world applications. Recently, deep neural models are gaining increasing popularity for tex…
BlockingClassificationFeature EngineeringGeneral Classification+2LIME: Weakly-Supervised Text Classification Without Seeds
In weakly-supervised text classification, only label names act as sources of supervision. Predominant approaches to weakly-supervised text classification utilize a two-phase framework, where test samples are first assign…
ClassificationNatural Language Inferencetext-classificationText Classification+1Weakly Supervised Veracity Classification with LLM-Predicted Credibility Signals
Credibility signals represent a wide range of heuristics typically used by journalists and fact-checkers to assess the veracity of online content. Automating the extraction of credibility signals presents significant cha…
MisinformationVeracity Classification