Revisiting Over-smoothing in Deep GCNs

2020-03-30 · Chaoqi Yang, Ruijie Wang, Shuochao Yao, Shengzhong Liu, Tarek Abdelzaher

Oversmoothing has been assumed to be the major cause of performance drop in deep graph convolutional networks (GCNs). In this paper, we propose a new view that deep GCNs can actually learn to anti-oversmooth during training. This work interprets a standard GCN architecture as layerwise integration of a Multi-layer Perceptron (MLP) and graph regularization. We analyze and conclude that before training, the final representation of a deep GCN does over-smooth, however, it learns anti-oversmoothing during training. Based on the conclusion, the paper further designs a cheap but effective trick to improve GCN training. We verify our conclusions and evaluate the trick on three citation networks and further provide insights on neighborhood aggregation in GCNs.

📄 PDF Abstract BibTeX arXiv:2003.13663

Code (0)

등록된 구현이 없습니다.

Tasks

Node Classification

Methods 이 논문이 사용한 방법론

Graph Convolutional Networks 설명 없음

GCN A Graph Convolutional Network, or GCN, is an approach for semi-supervised learning on graph-structured data. It is based on an efficient variant of [convolutional neural…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Graph Neural Networks Do Not Always Oversmooth

2024-06-04 · Bastian Epping, Alexandre René, Moritz Helias, Michael T. Schaub

Graph neural networks (GNNs) have emerged as powerful tools for processing relational data in applications. However, GNNs suffer from the problem of oversmoothing, the property that the features of all nodes exponentiall…

Multi-scale Graph Convolutional Networks with Self-Attention

2021-12-04 · Zhilong Xiong, Jia Cai

Graph convolutional networks (GCNs) have achieved remarkable learning ability for dealing with various graph structural data recently. In general, deep GCNs do not work well since graph convolution in conventional GCNs i…

Computational Efficiency Graph Classification Node Classification

Revisiting Graph Convolutional Network on Semi-Supervised Node Classification from an Optimization Perspective

2020-09-24 · Hongwei Zhang, Tijin Yan, Zenjun Xie, Yuanqing Xia 외

Graph convolutional networks (GCNs) have achieved promising performance on various graph-based tasks. However they suffer from over-smoothing when stacking more layers. In this paper, we present a quantitative study on t…

General Classification Node Classification

On the Global Self-attention Mechanism for Graph Convolutional Networks

2020-10-21 · Chen Wang, Chengyuan Deng

Applying Global Self-attention (GSA) mechanism over features has achieved remarkable success on Convolutional Neural Networks (CNNs). However, it is not clear if Graph Convolutional Networks (GCNs) can similarly benefit …

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks

2024-07-01 · Guangrui Yang, Jianfei Li, Ming Li, Han Feng 외

In this paper, we explore the approximation theory of functions defined on graphs. Our study builds upon the approximation results derived from the $K$-functional. We establish a theoretical framework to assess the lower…