Dimensionality Reduction Meets Message Passing for Graph Node Embeddings
Graph Neural Networks (GNNs) have become a popular approach for various applications, ranging from social network analysis to modeling chemical properties of molecules. While GNNs often show remarkable performance on public datasets, they can struggle to learn long-range dependencies in the data due to over-smoothing and over-squashing tendencies. To alleviate this challenge, we propose PCAPass, a method which combines Principal Component Analysis (PCA) and message passing for generating node embeddings in an unsupervised manner and leverages gradient boosted decision trees for classification tasks. We show empirically that this approach provides competitive performance compared to popular GNNs on node classification benchmarks, while gathering information from longer distance neighborhoods. Our research demonstrates that applying dimensionality reduction with message passing and skip connections is a promising mechanism for aggregating long-range dependencies in graph structured data.
Code (1)
Tasks
Dimensionality ReductionNode ClassificationNode Property PredictionSimilar Papers 제목 키워드 기반
Semi-Supervised Graph Learning Meets Dimensionality Reduction
Semi-supervised learning (SSL) has recently received increased attention from machine learning researchers. By enabling effective propagation of known labels in graph-based deep learning (GDL) algorithms, SSL is poised t…
ClusteringDimensionality ReductionGraph LearningNode ClusteringMessage Passing Meets Graph Neural Networks: A New Paradigm for Massive MIMO Systems
As one of the core technologies for 5G systems, massive multiple-input multiple-output (MIMO) introduces dramatic capacity improvements along with very high beamforming and spatial multiplexing gains. When developing eff…
Graph Neural NetworkMixture of Experts Meets Decoupled Message Passing: Towards General and Adaptive Node Classification
Graph neural networks excel at graph representation learning but struggle with heterophilous data and long-range dependencies. And graph transformers address these issues through self-attention, yet face scalability and …
Computational EfficiencyGraph Representation LearningMixture-of-Experts+3Neural Message Passing on Structural Interaction Graphs for Fully-Inductive Graph Neural Networks
A central obstacle in building graph foundation models is the input heterogeneity in terms of feature space dimensionality, semantics, and structure. Such heterogeneity limits the capability of graph neural networks to g…
Inductive Link PredictionStein Variational Message Passing for Continuous Graphical Models
We propose a novel distributed inference algorithm for continuous graphical models, by extending Stein variational gradient descent (SVGD) to leverage the Markov dependency structure of the distribution of interest. Our …