On Data Augmentation for Extreme Multi-label Classification

2020-09-22 · Danqing Zhang, Tao Li, Haiyang Zhang, Bing Yin

In this paper, we focus on data augmentation for the extreme multi-label classification (XMC) problem. One of the most challenging issues of XMC is the long tail label distribution where even strong models suffer from insufficient supervision. To mitigate such label bias, we propose a simple and effective augmentation framework and a new state-of-the-art classifier. Our augmentation framework takes advantage of the pre-trained GPT-2 model to generate label-invariant perturbations of the input texts to augment the existing training data. As a result, it present substantial improvements over baseline models. Our contributions are two-factored: (1) we introduce a new state-of-the-art classifier that uses label attention with RoBERTa and combine it with our augmentation framework for further improvement; (2) we present a broad study on how effective are different augmentation methods in the XMC task.

📄 PDF Abstract BibTeX arXiv:2009.10778

Code (0)

등록된 구현이 없습니다.

Tasks

Classification Data AugmentationExtreme Multi-Label ClassificationGeneral Classification Multi-Label Classification MUlTI-LABEL-ClASSIFICATION

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Cosine Annealing Cosine Annealing is a type of learning rate schedule that has the effect of starting with a large learning rate that is relatively rapidly decreased to a minimum value before…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

Linear Warmup With Linear Decay Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for $n$ updates and then linearly decay afterwards.

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Weight Decay 설명 없음

Linear Warmup With Cosine Annealing Linear Warmup With Cosine Annealing is a learning rate schedule where we increase the learning rate linearly for $n$ updates and then anneal according to a cosine schedule…

DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification

2018-08-29 · Xiaofeng Zhang, Zhangyang Wang, Dong Liu, Qing Ling

Deep learning has revolutionized the performance of classification, but meanwhile demands sufficient labeled data for training. Given insufficient data, while many techniques have been developed to help combat overfittin…

Data Augmentation General Classification Generative Adversarial Network Transfer Learning

TailMix: Overcoming the Label Sparsity for Extreme Multi-label Classification

2021-09-29 · Sangwoo Han, Chan Lim, Jongwuk Lee

Extreme multi-label classification (XMC) aims at finding the most relevant labels from a huge label set at the industrial scale. The XMC problem inherently poses two challenges: data scalability and label sparsity. This …

Extreme Multi-Label Classification Multi-Label Classification MUlTI-LABEL-ClASSIFICATION

MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification

2020-04-25 · ACL 2020 6 · Jiaao Chen, Zichao Yang, Diyi Yang

This paper presents MixText, a semi-supervised learning method for text classification, which uses our newly designed data augmentation method called TMix. TMix creates a large amount of augmented training samples by int…

Classification Data Augmentation General Classification Semi-Supervised Text Classification+1

Application of Deep Learning Methods to SNOMED CT Encoding of Clinical Texts: From Data Collection to Extreme Multi-Label Text-Based Classification

2021-09-01 · RANLP 2021 9 · Anton Hristov, Aleksandar Tahchiev, Hristo Papazov, Nikola Tulechki 외

Concept normalization of clinical texts to standard medical classifications and ontologies is a task with high importance for healthcare and medical research. We attempt to solve this problem through automatic SNOMED CT …

Classification Data Augmentation Dataset Generation

Adopting the Multi-answer Questioning Task with an Auxiliary Metric for Extreme Multi-label Text Classification Utilizing the Label Hierarchy

2023-03-02 · Li Wang, Ying Wah Teh, Mohammed Ali Al-Garadi

Extreme multi-label text classification utilizes the label hierarchy to partition extreme labels into multiple label groups, turning the task into simple multi-group multi-label classification tasks. Current research enc…

Classification Extreme Multi-Label Classification Multi-Label Classification MUlTI-LABEL-ClASSIFICATION+5