SkipNode: On Alleviating Performance Degradation for Deep Graph Convolutional Networks
Graph Convolutional Networks (GCNs) suffer from performance degradation when models go deeper. However, earlier works only attributed the performance degeneration to over-smoothing. In this paper, we conduct theoretical and experimental analysis to explore the fundamental causes of performance degradation in deep GCNs: over-smoothing and gradient vanishing have a mutually reinforcing effect that causes the performance to deteriorate more quickly in deep GCNs. On the other hand, existing anti-over-smoothing methods all perform full convolutions up to the model depth. They could not well resist the exponential convergence of over-smoothing due to model depth increasing. In this work, we propose a simple yet effective plug-and-play module, Skipnode, to overcome the performance degradation of deep GCNs. It samples graph nodes in each convolutional layer to skip the convolution operation. In this way, both over-smoothing and gradient vanishing can be effectively suppressed since (1) not all nodes'features propagate through full layers and, (2) the gradient can be directly passed back through ``skipped'' nodes. We provide both theoretical analysis and empirical evaluation to demonstrate the efficacy of Skipnode and its superiority over SOTA baselines.
Code (1)
Tasks
Link PredictionNode ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Towards Cross-Disaster Building Damage Assessment with Graph Convolutional Networks
In the aftermath of disasters, building damage maps are obtained using change detection to plan rescue operations. Current convolutional neural network approaches do not consider the similarities between neighboring buil…
Building Damage AssessmentChange DetectionUnderstanding and Resolving Performance Degradation in Graph Convolutional Networks
A Graph Convolutional Network (GCN) stacks several layers and in each layer performs a PROPagation operation (PROP) and a TRANsformation operation (TRAN) for learning node representations over graph-structured data. Thou…
Multi-Scale Spatio-Temporal Graph Convolutional Network for Facial Expression Spotting
Facial expression spotting is a significant but challenging task in facial expression analysis. The accuracy of expression spotting is affected not only by irrelevant facial movements but also by the difficulty of percei…
Contrastive LearningMicro-Expression SpottingLoop-Mamba: A Loop Mamba with Degradation-Aware and Shared Memory for Old Photo Restoration
Old photographs often suffer from multiple coupled degradations, including scratches, cracks, fading, blur, noise, and missing regions, severely degrading both visual quality and semantic content. We propose Loop-Mamba, …
Efficient Traffic Prediction Through Spatio-Temporal Distillation
Graph neural networks (GNNs) have gained considerable attention in recent years for traffic flow prediction due to their ability to learn spatio-temporal pattern representations through a graph-based message-passing fram…
Knowledge DistillationPredictionTraffic Prediction