Tackling Over-Smoothing for General Graph Convolutional Networks
Increasing the depth of GCN, which is expected to permit more expressivity, is shown to incur performance detriment especially on node classification. The main cause of this lies in over-smoothing. The over-smoothing issue drives the output of GCN towards a space that contains limited distinguished information among nodes, leading to poor expressivity. Several works on refining the architecture of deep GCN have been proposed, but it is still unknown in theory whether or not these refinements are able to relieve over-smoothing. In this paper, we first theoretically analyze how general GCNs act with the increase in depth, including generic GCN, GCN with bias, ResGCN, and APPNP. We find that all these models are characterized by a universal process: all nodes converging to a cuboid. Upon this theorem, we propose DropEdge to alleviate over-smoothing by randomly removing a certain number of edges at each training epoch. Theoretically, DropEdge either reduces the convergence speed of over-smoothing or relieves the information loss caused by dimension collapse. Experimental evaluations on simulated dataset have visualized the difference in over-smoothing between different GCNs. Moreover, extensive experiments on several real benchmarks support that DropEdge consistently improves the performance on a variety of both shallow and deep GCNs.
Code (0)
등록된 구현이 없습니다.
Tasks
Node ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Graph Partner Neural Networks for Semi-Supervised Learning on Graphs
Graph Convolutional Networks (GCNs) are powerful for processing graph-structured data and have achieved state-of-the-art performance in several tasks such as node classification, link prediction, and graph classification…
ClassificationGraph ClassificationLink PredictionNode ClassificationSStaGCN: Simplified stacking based graph convolutional networks
Graph convolutional network (GCN) is a powerful model studied broadly in various graph structural data learning tasks. However, to mitigate the over-smoothing phenomenon, and deal with heterogeneous graph structural data…
VEM-GCN: Topology Optimization with Variational EM for Graph Convolutional Networks
Over-smoothing has emerged as a severe problem for node classification with graph convolutional networks (GCNs). In the view of message passing, the over-smoothing issue is caused by the observed noisy graph topology tha…
ClassificationGeneral ClassificationNode ClassificationStochastic Block ModelATNPA: A Unified View of Oversmoothing Alleviation in Graph Neural Networks
Oversmoothing is a commonly observed challenge in graph neural network (GNN) learning, where, as layers increase, embedding features learned from GNNs quickly become similar/indistinguishable, making them incapable of di…
Graph Neural NetworkRelationMamba-Based Graph Convolutional Networks: Tackling Over-smoothing with Selective State Space
Graph Neural Networks (GNNs) have shown great success in various graph-based learning tasks. However, it often faces the issue of over-smoothing as the model depth increases, which causes all node representations to conv…
Graph Neural NetworkGraph Representation LearningMambaNode Classification+1