Message Passing Attention Networks for Document Understanding
Graph neural networks have recently emerged as a very effective framework for processing graph-structured data. These models have achieved state-of-the-art performance in many tasks. Most graph neural networks can be described in terms of message passing, vertex update, and readout functions. In this paper, we represent documents as word co-occurrence networks and propose an application of the message passing framework to NLP, the Message Passing Attention network for Document understanding (MPAD). We also propose several hierarchical variants of MPAD. Experiments conducted on 10 standard text classification datasets show that our architectures are competitive with the state-of-the-art. Ablation studies reveal further insights about the impact of the different components on performance. Code is publicly available at: https://github.com/giannisnik/mpad .
Code (2)
Tasks
document understandingMulti-Modal Document Classificationtext-classificationText ClassificationSimilar Papers 제목 키워드 기반
A Message Passing Perspective on Learning Dynamics of Contrastive Learning
In recent years, contrastive learning achieves impressive results on self-supervised visual representation learning, but there still lacks a rigorous understanding of its learning dynamics. In this paper, we show that if…
Contrastive LearningGraph AttentionRepresentation LearningUnderstanding the Message Passing in Graph Neural Networks via Power Iteration Clustering
The mechanism of message passing in graph neural networks (GNNs) is still mysterious. Apart from convolutional neural networks, no theoretical origin for GNNs has been proposed. To our surprise, message passing can be be…
ClusteringMulti-hop Reading Comprehension across Multiple Documents by Reasoning over Heterogeneous Graphs
Multi-hop reading comprehension (RC) across documents poses new challenge over single-document RC because it requires reasoning over multiple documents to reach the final answer. In this paper, we propose a new model to …
Multi-Hop Reading ComprehensionReading ComprehensionGPS++: Reviving the Art of Message Passing for Molecular Property Prediction
We present GPS++, a hybrid Message Passing Neural Network / Graph Transformer model for molecular property prediction. Our model integrates a well-tuned local message passing component and biased global attention with ot…
Molecular Property PredictionProperty PredictionConvergence of Message Passing Graph Neural Networks with Generic Aggregation On Large Random Graphs
We study the convergence of message passing graph neural networks on random graph models to their continuous counterpart as the number of nodes tends to infinity. Until now, this convergence was only known for architectu…