Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks
Before entering the neural network, a token needs to be converted to its one-hot representation, which is a discrete distribution of the vocabulary. Smoothed representation is the probability of candidate tokens obtained from the pre-trained masked language model, which can be seen as a more informative augmented substitution to the one-hot representation. We propose an efficient data augmentation method, dub as text smoothing, by converting a sentence from its one-hot representation to controllable smoothed representation. We evaluate text smoothing on different datasets in a low-resource regime. Experimental results show that text smoothing outperforms various mainstream data augmentation methods by a substantial margin. Moreover, text smoothing can be combined with these data augmentation methods to achieve better performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationLanguage ModelingLanguage ModellingSentencetext-classificationText ClassificationSimilar Papers 제목 키워드 기반
Text Smoothing: Enhance Various Data Augmentation Methods on Text Classification Tasks
Before entering the neural network, a token is generally converted to the corresponding one-hot representation, which is a discrete distribution of the vocabulary. Smoothed representation is the probability of candidate …
Data AugmentationLanguage ModelingLanguage ModellingSentence+2Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks
Text classification tasks often encounter few shot scenarios with limited labeled data, and addressing data scarcity is crucial. Data augmentation with mixup has shown to be effective on various text classification tasks…
ClassificationData AugmentationFew-Shot Text Classificationtext-classification+1Revisiting the Role of Label Smoothing in Enhanced Text Sentiment Classification
Label smoothing is a widely used technique in various domains, such as text classification, image classification and speech recognition, known for effectively combating model overfitting. However, there is little fine-gr…
Classificationimage-classificationImage ClassificationSentiment Analysis+5Dirichlet Energy Enhancement of Graph Neural Networks by Framelet Augmentation
Graph convolutions have been a pivotal element in learning graph representations. However, recursively aggregating neighboring information with graph convolutions leads to indistinguishable node features in deep layers, …
Node ClassificationAutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes
Text data augmentation is a complex problem due to the discrete nature of sentences. Although rule-based augmentation methods are widely adopted in real-world applications because of their simplicity, they suffer from po…
Data Augmentation