paper-with-me

Papers

A Simple Graph Contrastive Learning Framework for Short Text Classification

2025-01-16 · Yonghao Liu, Fausto Giunchiglia, Lan Huang, Ximing Li, Xiaoyue Feng, Renchu Guan

Short text classification has gained significant attention in the information age due to its prevalence and real-world applications. Recent advancements in graph learning combined with contrastive learning have shown promising results in addressing the challenges of semantic sparsity and limited labeled data in short text classification. However, existing models have certain limitations. They rely on explicit data augmentation techniques to generate contrastive views, resulting in semantic corruption and noise. Additionally, these models only focus on learning the intrinsic consistency between the generated views, neglecting valuable discriminative information from other potential views. To address these issues, we propose a Simple graph contrastive learning framework for Short Text Classification (SimSTC). Our approach involves performing graph learning on multiple text-related component graphs to obtain multi-view text embeddings. Subsequently, we directly apply contrastive learning on these embeddings. Notably, our method eliminates the need for data augmentation operations to generate contrastive views while still leveraging the benefits of multi-view contrastive learning. Despite its simplicity, our model achieves outstanding performance, surpassing large language models on various datasets.

📄 PDF Abstract BibTeX arXiv:2501.09219

Code (1)

keaml-jlu/simstc 공식 구현 pytorch

Tasks

Contrastive LearningData AugmentationGraph Learningtext-classificationText Classification

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음
Contrastive Learning 설명 없음
Focus 설명 없음

Similar Papers 제목 키워드 기반

Unifying Graph Contrastive Learning with Flexible Contextual Scopes

2022-10-17 · Yizhen Zheng, Yu Zheng, Xiaofei Zhou, Chen Gong 외

Graph contrastive learning (GCL) has recently emerged as an effective learning paradigm to alleviate the reliance on labelling information for graph representation learning. The core of GCL is to maximise the mutual info…

Contrastive LearningGraph Representation LearningRepresentation LearningSelf-Supervised Learning

Demonstrating and Reducing Shortcuts in Vision-Language Representation Learning

2024-02-27 · Maurits Bleeker, Mariya Hendriksen, Andrew Yates, Maarten de Rijke

Vision-language models (VLMs) mainly rely on contrastive training to learn general-purpose representations of images and captions. We focus on the situation when one image is associated with several captions, each captio…

Contrastive LearningRepresentation Learning

Contrastive Document Representation Learning with Graph Attention Networks

2021-10-20 · Findings (EMNLP) 2021 11 · Peng Xu, Xinchi Chen, Xiaofei Ma, Zhiheng Huang 외

Recent progress in pretrained Transformer-based language models has shown great success in learning contextual representation of text. However, due to the quadratic self-attention complexity, most of the pretrained Trans…

Contrastive LearningDocument ClassificationGraph AttentionRepresentation Learning+1

Neighbour Contrastive Learning with Heterogeneous Graph Attention Networks on Short Text Classification

2021-11-16 · ACL ARR November 2021 11 · Anonymous

Graph neural networks (GNNs) have attracted extensive research interests in text classification tasks, due to their superiority in representation learning. However, most existing studies adopt the same semi-supervised le…

Contrastive LearningGraph AttentionGraph Neural NetworkRepresentation Learning+2

Localized Contrastive Learning on Graphs

2022-12-08 · Hengrui Zhang, Qitian Wu, Yu Wang, Shaofeng Zhang 외

Contrastive learning methods based on InfoNCE loss are popular in node representation learning tasks on graph-structured data. However, its reliance on data augmentation and its quadratic computational complexity might l…

Contrastive LearningData AugmentationRepresentation Learning