Infinite Width Graph Neural Networks for Node Regression/ Classification
This work analyzes Graph Neural Networks, a generalization of Fully-Connected Deep Neural Nets on Graph structured data, when their width, that is the number of nodes in each fullyconnected layer is increasing to infinity. Infinite Width Neural Networks are connecting Deep Learning to Gaussian Processes and Kernels, both Machine Learning Frameworks with long traditions and extensive theoretical foundations. Gaussian Processes and Kernels have much less hyperparameters then Neural Networks and can be used for uncertainty estimation, making them more user friendly for applications. This works extends the increasing amount of research connecting Gaussian Processes and Kernels to Neural Networks. The Kernel and Gaussian Process closed forms are derived for a variety of architectures, namely the standard Graph Neural Network, the Graph Neural Network with Skip-Concatenate Connections and the Graph Attention Neural Network. All architectures are evaluated on a variety of datasets on the task of transductive Node Regression and Classification. Additionally, a Spectral Sparsification method known as Effective Resistance is used to improve runtime and memory requirements. Extending the setting to inductive graph learning tasks (Graph Regression/ Classification) is straightforward and is briefly discussed in 3.5.
Code (1)
Tasks
Gaussian ProcessesGraph AttentionGraph LearningGraph Neural NetworkGraph RegressionNode RegressionregressionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Flexible infinite-width graph convolutional networks and the importance of representation learning
A common theoretical approach to understanding neural networks is to take an infinite-width limit, at which point the outputs become Gaussian process (GP) distributed. This is known as a neural network Gaussian process (…
ClassificationGraph ClassificationNode ClassificationRepresentation LearningInfinitely Wide Graph Convolutional Networks: Semi-supervised Learning via Gaussian Processes
Graph convolutional neural networks~(GCNs) have recently demonstrated promising results on graph-based semi-supervised classification, but little work has been done to explore their theoretical properties. Recently, seve…
Gaussian ProcessesGeneral ClassificationUncertainty Quantification From Scaling Laws in Deep Neural Networks
Quantifying the uncertainty from machine learning analyses is critical to their use in the physical sciences. In this work we focus on uncertainty inherited from the initialization distribution of neural networks. We com…
Uncertainty QuantificationGaussian Process Limit Reveals Structural Benefits of Graph Transformers
Graph transformers are the state-of-the-art for learning from graph-structured data and are empirically known to avoid several pitfalls of message-passing architectures. However, there is limited theoretical analysis on …
Is Solving Graph Neural Tangent Kernel Equivalent to Training Graph Neural Network?
A rising trend in theoretical deep learning is to understand why deep learning works through Neural Tangent Kernel (NTK) [jgh18], a kernel method that is equivalent to using gradient descent to train a multi-layer infini…
Deep LearningGraph LearningGraph Neural Networkregression