Towards Feature Overcorrelation in Deeper Graph Neural Networks
Graph neural networks (GNNs) have achieved great success in graph representation learning, which has tremendously facilitated various real-world applications. Nevertheless, the performance of GNNs significantly deteriorates when the depth increases. Recent researches have attributed this phenomenon to the oversmoothing issue, which indicates that the learned node representations are highly indistinguishable. In this paper, we observe a new issue in deeper GNNs, i.e., feature overcorrelation, and perform a thorough study to deepen our understanding on this issue. In particular, we demonstrate the existence of feature overcorrelation in deeper GNNs, reveal potential reasons leading to this issue, and validate that overcorrelation and oversmoothing present different patterns though they are related. Since feature overcorrelation indicates that GNNs encode less information and can harm the downstream tasks, it is of great significance to mitigate this issue. Therefore, we propose the DeCorr, a general framework to effectively reduce feature correlation for deeper GNNs. Experimental results on various datasets demonstrate that DeCorr can help train deeper GNNs effectively and is complementary to methods tackling oversmoothing.
Code (0)
등록된 구현이 없습니다.
Tasks
Feature CorrelationGraph Representation LearningRepresentation LearningSimilar Papers 제목 키워드 기반
Feature Overcorrelation in Deep Graph Neural Networks: A New Perspective
Recent years have witnessed remarkable success achieved by graph neural networks (GNNs) in many real-world applications such as recommendation and drug discovery. Despite the success, oversmoothing has been identified as…
Drug DiscoveryFeature CorrelationDeeper-GXX: Deepening Arbitrary GNNs
Recently, motivated by real applications, a major research direction in graph neural networks (GNNs) is to explore deeper structures. For instance, the graph connectivity is not always consistent with the label distribut…
Contrastive LearningLink PredictionNode ClassificationRepresentation LearningResponse to Promises and Pitfalls of Deep Kernel Learning
This note responds to "Promises and Pitfalls of Deep Kernel Learning" (Ober et al., 2021). The marginal likelihood of a Gaussian process can be compartmentalized into a data fit term and a complexity penalty. Ober et al.…
Graph DNA: Deep Neighborhood Aware Graph Encoding for Collaborative Filtering
In this paper, we consider recommender systems with side information in the form of graphs. Existing collaborative filtering algorithms mainly utilize only immediate neighborhood information and have a hard time taking a…
Collaborative FilteringRecommendation SystemsDark Spot Detection from SAR Images Based on Superpixel Deeper Graph Convolutional Network
Synthetic Aperture Radar (SAR) is the main instrument utilized for the detection of oil slicks on the ocean surface. In SAR images, some areas affected by ocean phenomena, such as rain cells, upwellings, and internal wav…
Graph Neural NetworkNode ClassificationSemantic SegmentationSuperpixels