PairNorm: Tackling Oversmoothing in GNNs
The performance of graph neural nets (GNNs) is known to gradually decrease with increasing number of layers. This decay is partly attributed to oversmoothing, where repeated graph convolutions eventually make node embeddings indistinguishable. We take a closer look at two different interpretations, aiming to quantify oversmoothing. Our main contribution is PairNorm, a novel normalization layer that is based on a careful analysis of the graph convolution operator, which prevents all node embeddings from becoming too similar. What is more, PairNorm is fast, easy to implement without any change to network architecture nor any additional parameters, and is broadly applicable to any GNN. Experiments on real-world graphs demonstrate that PairNorm makes deeper GCN, GAT, and SGC models more robust against oversmoothing, and significantly boosts performance for a new problem setting that benefits from deeper GNNs. Code is available at https://github.com/LingxiaoShawn/PairNorm.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
ATNPA: A Unified View of Oversmoothing Alleviation in Graph Neural Networks
Oversmoothing is a commonly observed challenge in graph neural network (GNN) learning, where, as layers increase, embedding features learned from GNNs quickly become similar/indistinguishable, making them incapable of di…
Graph Neural NetworkRelationTowards Feature Overcorrelation in Deeper Graph Neural Networks
Graph neural networks (GNNs) have achieved great success in graph representation learning, which has tremendously facilitated various real-world applications. Nevertheless, the performance of GNNs significantly deteriora…
Feature CorrelationGraph Representation LearningRepresentation LearningFeature Overcorrelation in Deep Graph Neural Networks: A New Perspective
Recent years have witnessed remarkable success achieved by graph neural networks (GNNs) in many real-world applications such as recommendation and drug discovery. Despite the success, oversmoothing has been identified as…
Drug DiscoveryFeature CorrelationThe Oversmoothing Fallacy: A Misguided Narrative in GNN Research
Oversmoothing has been recognized as a main obstacle to building deep Graph Neural Networks (GNNs), limiting the performance. This position paper argues that the influence of oversmoothing has been overstated and advocat…
MisconceptionsBackward Oversmoothing: why is it hard to train deep Graph Neural Networks?
Oversmoothing has long been identified as a major limitation of Graph Neural Networks (GNNs): input node features are smoothed at each layer and converge to a non-informative representation, if the weights of the GNN are…