Enhancing Chinese Multi-Label Text Classification Performance with Response-based Knowledge Distillation
It’s difficult to optimize individual label performance of multi-label text classification, especially in those imbalanced data containing long-tailed labels. Therefore, this study proposes a response-based knowledge distillation mechanism comprising a teacher model that optimizes binary classifiers of the corresponding labels and a student model that is a standalone multi-label classifier learning from distilled knowledge passed by the teacher model. A total of 2,724 Chinese healthcare texts were collected and manually annotated across nine defined labels, resulting in 8731 labels, each containing an average of 3.2 labels. We used 5-fold cross-validation to compare the performance of several multi-label models, including TextRNN, TextCNN, HAN, and GRU-att. Experimental results indicate that using the proposed knowledge distillation mechanism effectively improved the performance no matter which model was used, about 2-3% of micro-F1, 4-6% of macro-F1, 3-4% of weighted-F1 and 1-2% of subset accuracy for performance enhancement.
Code (0)
등록된 구현이 없습니다.
Tasks
Knowledge DistillationMulti Label Text ClassificationMulti-Label Text Classificationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Multi-Label Classification of Chinese Humor Texts Using Hypergraph Attention Networks
We use Hypergraph Attention Networks (HyperGAT) to recognize multiple labels of Chinese humor texts. We firstly represent a joke as a hypergraph. The sequential hyperedge and semantic hyperedge structures are used to con…
Multi-Label ClassificationMUlTI-LABEL-ClASSIFICATIONMoto: Enhancing Embedding with Multiple Joint Factors for Chinese Text Classification
Recently, language representation techniques have achieved great performances in text classification. However, most existing representation models are specifically designed for English materials, which may fail in Chines…
text-classificationText ClassificationImbalanced Chinese Multi-label Text Classification Based on Alternating Attention
The Design and Construction of a Chinese Sarcasm Dataset
As a typical multi-layered semi-conscious language phenomenon, sarcasm is widely existed in social media text for enhancing the emotion expression. Thus, the detection and processing of sarcasm is important to social med…