Constructing Contrastive samples via Summarization for Text Classification with limited annotations
Contrastive Learning has emerged as a powerful representation learning method and facilitates various downstream tasks especially when supervised data is limited. How to construct efficient contrastive samples through data augmentation is key to its success. Unlike vision tasks, the data augmentation method for contrastive learning has not been investigated sufficiently in language tasks. In this paper, we propose a novel approach to construct contrastive samples for language tasks using text summarization. We use these samples for supervised contrastive learning to gain better text representations which greatly benefit text classification tasks with limited annotations. To further improve the method, we mix up samples from different classes and add an extra regularization, named Mixsum, in addition to the cross-entropy-loss. Experiments on real-world text classification datasets (Amazon-5, Yelp-5, AG News, and IMDb) demonstrate the effectiveness of the proposed contrastive learning framework with summarization-based data augmentation and Mixsum regularization.
Code (1)
Tasks
Contrastive LearningData AugmentationGeneral ClassificationRepresentation Learningtext-classificationText ClassificationText SummarizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
CaPE: Contrastive Parameter Ensembling for Reducing Hallucination in Abstractive Summarization
Hallucination is a known issue for neural abstractive summarization models. Recent work suggests that the degree of hallucination may depend on errors in the training data. In this work, we propose a new method called Co…
Abstractive Text SummarizationARCHallucinationInformativenessAlign and Attend: Multimodal Summarization with Dual Contrastive Losses
The goal of multimodal summarization is to extract the most important information from different modalities to form output summaries. Unlike the unimodal summarization, the multimodal summarization task explicitly levera…
Extractive Text SummarizationSupervised Video SummarizationVideo SummarizationFrequency-Masked Embedding Inference: A Non-Contrastive Approach for Time Series Representation Learning
Contrastive learning underpins most current self-supervised time series representation methods. The strategy for constructing positive and negative sample pairs significantly affects the final representation quality. How…
Contrastive LearningLinear evaluationRepresentation LearningTime SeriesContrastive Data and Learning for Natural Language Processing
Current NLP models heavily rely on effective representation learning algorithms. Contrastive learning is one such technique to learn an embedding space such that similar data sample pairs have close representations while…
Contrastive LearningQuestion AnsweringRepresentation LearningSentence+3ConCur: Self-supervised graph representation based on contrastive learning with curriculum negative sampling
Contrastive learning has made breakthrough advancements in graph representation learning, which encourages the representation of positive samples to be close and those of negative samples to be far away. However, existin…
Contrastive LearningGraph Representation LearningNode ClassificationRepresentation Learning+1