Comment on "A Note on Over-Smoothing for Graph Neural Networks"
We comment on Cai and Wang (2020, arXiv:2006.13318), who analyze over-smoothing in GNNs via Dirichlet energy. We show that under mild spectral conditions (including with Leaky-ReLU), the Dirichlet energy of node embeddings decreases exponentially with depth; we further extend the result to spectral polynomial filters and provide a short proof for the Leaky-ReLU case. Experiments on edge deletion and weight amplification illustrate when Dirichlet energy increases, hinting at practical ways to relieve over-smoothing.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
A Note on Over-Smoothing for Graph Neural Networks
Graph Neural Networks (GNNs) have achieved a lot of success on graph-structured data. However, it is observed that the performance of graph neural networks does not improve as the number of layers increases. This effect,…
Graph Neural NetworkDigging Up Citations: FOSSIL, a Dataset and Workflow for Reference Extraction in Law and the Humanities
Citation extraction tools are designed for the structured end-of-document bibliographies of the natural sciences, but law and humanities scholarship cites references primarily in footnotes, where bibliographic data is in…
Creating Corpora for Research in Feedback Comment Generation
In this paper, we report on datasets that we created for research in feedback comment generation {---} a task of automatically generating feedback comments such as a hint or an explanatory note for writing learning. Ther…
Comment GenerationLong-range Meta-path Search on Large-scale Heterogeneous Graphs
Utilizing long-range dependency, a concept extensively studied in homogeneous graphs, remains underexplored in heterogeneous graphs, especially on large ones, posing two significant challenges: Reducing computational cos…
Node ClassificationNode Property PredictionPredicting the Understandability of Computational Notebooks through Code Metrics Analysis
Computational notebooks are the primary coding tools for data scientists, but their code quality remains understudied and often poor. Given the importance of maintainability and reusability, enhancing code understandabil…