A Simple Graph Contrastive Learning Framework for Short Text Classification
Short text classification has gained significant attention in the information age due to its prevalence and real-world applications. Recent advancements in graph learning combined with contrastive learning have shown promising results in addressing the challenges of semantic sparsity and limited labeled data in short text classification. However, existing models have certain limitations. They rely on explicit data augmentation techniques to generate contrastive views, resulting in semantic corruption and noise. Additionally, these models only focus on learning the intrinsic consistency between the generated views, neglecting valuable discriminative information from other potential views. To address these issues, we propose a Simple graph contrastive learning framework for Short Text Classification (SimSTC). Our approach involves performing graph learning on multiple text-related component graphs to obtain multi-view text embeddings. Subsequently, we directly apply contrastive learning on these embeddings. Notably, our method eliminates the need for data augmentation operations to generate contrastive views while still leveraging the benefits of multi-view contrastive learning. Despite its simplicity, our model achieves outstanding performance, surpassing large language models on various datasets.
Code (1)
Tasks
Contrastive LearningData AugmentationGraph Learningtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Unifying Graph Contrastive Learning with Flexible Contextual Scopes
Graph contrastive learning (GCL) has recently emerged as an effective learning paradigm to alleviate the reliance on labelling information for graph representation learning. The core of GCL is to maximise the mutual info…
Contrastive LearningGraph Representation LearningRepresentation LearningSelf-Supervised LearningDemonstrating and Reducing Shortcuts in Vision-Language Representation Learning
Vision-language models (VLMs) mainly rely on contrastive training to learn general-purpose representations of images and captions. We focus on the situation when one image is associated with several captions, each captio…
Contrastive LearningRepresentation LearningContrastive Document Representation Learning with Graph Attention Networks
Recent progress in pretrained Transformer-based language models has shown great success in learning contextual representation of text. However, due to the quadratic self-attention complexity, most of the pretrained Trans…
Contrastive LearningDocument ClassificationGraph AttentionRepresentation Learning+1Neighbour Contrastive Learning with Heterogeneous Graph Attention Networks on Short Text Classification
Graph neural networks (GNNs) have attracted extensive research interests in text classification tasks, due to their superiority in representation learning. However, most existing studies adopt the same semi-supervised le…
Contrastive LearningGraph AttentionGraph Neural NetworkRepresentation Learning+2Localized Contrastive Learning on Graphs
Contrastive learning methods based on InfoNCE loss are popular in node representation learning tasks on graph-structured data. However, its reliance on data augmentation and its quadratic computational complexity might l…
Contrastive LearningData AugmentationRepresentation Learning