Label-Wise Document Pre-Training for Multi-Label Text Classification
A major challenge of multi-label text classification (MLTC) is to stimulatingly exploit possible label differences and label correlations. In this paper, we tackle this challenge by developing Label-Wise Pre-Training (LW-PT) method to get a document representation with label-aware information. The basic idea is that, a multi-label document can be represented as a combination of multiple label-wise representations, and that, correlated labels always cooccur in the same or similar documents. LW-PT implements this idea by constructing label-wise document classification tasks and trains label-wise document encoders. Finally, the pre-trained label-wise encoder is fine-tuned with the downstream MLTC task. Extensive experimental results validate that the proposed method has significant advantages over the previous state-of-the-art models and is able to discover reasonable label relationship. The code is released to facilitate other researchers.
Code (1)
Tasks
ClassificationDocument ClassificationGeneral ClassificationMulti Label Text ClassificationMulti-Label Text Classificationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Pairwise Instance Relation Augmentation for Long-tailed Multi-label Text Classification
Multi-label text classification (MLTC) is one of the key tasks in natural language processing. It aims to assign multiple target labels to one document. Due to the uneven popularity of labels, the number of documents per…
DiversityMulti Label Text ClassificationMulti-Label Text ClassificationRelation+2Information retrieval for label noise document ranking by bag sampling and group-wise loss
Long Document retrieval (DR) has always been a tremendous challenge for reading comprehension and information retrieval. The pre-training model has achieved good results in the retrieval stage and Ranking for long docume…
Document RankingInformation RetrievalReading ComprehensionRetrievalRAPT: Retrieval-Augmented Post-hoc Thresholding for Multi-Label Classification
Industrial multi-label document understanding pipelines score candidate labels and threshold or rank them to form a label set per document. This early selection step directly affects the accuracy of downstream informatio…
Multi-Label ClassificationInformation ExtractionMetric LearningModeling Label Ambiguity for Neural List-Wise Learning to Rank
List-wise learning to rank methods are considered to be the state-of-the-art. One of the major problems with these methods is that the ambiguous nature of relevance labels in learning to rank data is ignored. Ambiguity o…
Learning-To-RankLearning a Concept Hierarchy from Multi-labeled Documents
While topic models can discover patterns of word usage in large corpora, it is difficult to meld this unsupervised structure with noisy, human-provided labels, especially when the label space is large. In this paper, we …
Missing LabelsTopic Models