Text Graph Transformer for Document Classification
Text classification is a fundamental problem in natural language processing. Recent studies applied graph neural network (GNN) techniques to capture global word co-occurrence in a corpus. However, previous works are not scalable to large-sized corpus and ignore the heterogeneity of the text graph. To address these problems, we introduce a novel Transformer based heterogeneous graph neural network, namely Text Graph Transformer (TG-Transformer). Our model learns effective node representations by capturing structure and heterogeneity from the text graph. We propose a mini-batch text graph sampling method that significantly reduces computing and memory costs to handle large-sized corpus. Extensive experiments have been conducted on several benchmark datasets, and the results demonstrate that TG-Transformer outperforms state-of-the-art approaches on text classification task.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationDocument ClassificationGeneral ClassificationGraph Neural NetworkGraph Samplingtext-classificationText ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Revisiting Transformer-based Models for Long Document Classification
The recent literature in text classification is biased towards short text sequences (e.g., sentences or paragraphs). In real-world applications, multi-page multi-paragraph documents are common and they cannot be efficien…
ClassificationDocument Classificationtext-classificationText ClassificationRevisiting Transformer-based Models for Long Document Classification
The recent literature in text classification is biased towards short text sequences (e.g., sentences or paragraphs). In real-world applications, multi-page multi-paragraph documents are common and they cannot be efficie…
ClassificationDocument Classificationtext-classificationText ClassificationContrastive Document Representation Learning with Graph Attention Networks
Recent progress in pretrained Transformer-based language models has shown great success in learning contextual representation of text. However, due to the quadratic self-attention complexity, most of the pretrained Trans…
Contrastive LearningDocument ClassificationGraph AttentionRepresentation Learning+1Connecting the Dots: What Graph-Based Text Representations Work Best for Text Classification Using Graph Neural Networks?
Given the success of Graph Neural Networks (GNNs) for structure-aware machine learning, many studies have explored their use for text classification, but mostly in specific domains with limited data characteristics. More…
graph constructionGraph Miningtext-classificationText ClassificationModTGCN: Modularity-aware Graph Neural Networks for Text Classification
Graph-based text classification models typically rely on local neighborhood aggregation and overlook global community structure, despite semantic document graphs exhibiting strong class-consistent clustering. Ignoring th…
Graph Neural NetworkText Classification