GNN-VPA: A Variance-Preserving Aggregation Strategy for Graph Neural Networks
Graph neural networks (GNNs), and especially message-passing neural networks, excel in various domains such as physics, drug discovery, and molecular modeling. The expressivity of GNNs with respect to their ability to discriminate non-isomorphic graphs critically depends on the functions employed for message aggregation and graph-level readout. By applying signal propagation theory, we propose a variance-preserving aggregation function (VPA) that maintains expressivity, but yields improved forward and backward dynamics. Experiments demonstrate that VPA leads to increased predictive performance for popular GNN architectures as well as improved learning dynamics. Our results could pave the way towards normalizer-free or self-normalizing GNNs.
Code (1)
Tasks
Drug DiscoverySimilar Papers 제목 키워드 기반
Graph Attribute Aggregation Network with Progressive Margin Folding
Graph convolutional neural networks (GCNNs) have been attracting increasing research attention due to its great potential in inference over graph structures. However, insufficient effort has been devoted to the aggregati…
AttributeOrientation covariant aggregation of local descriptors with embeddings
Image search systems based on local descriptors typically achieve orientation invariance by aligning the patches on their dominant orientations. Albeit successful, this choice introduces too much invariance because it do…
Image RetrievalRetrievalBreaking the Expressive Bottlenecks of Graph Neural Networks
Recently, the Weisfeiler-Lehman (WL) graph isomorphism test was used to measure the expressiveness of graph neural networks (GNNs), showing that the neighborhood aggregation GNNs were at most as powerful as 1-WL test in …
Graph Property PredictionFedGraph: an Aggregation Method from Graph Perspective
With the increasingly strengthened data privacy act and the difficult data centralization, Federated Learning (FL) has become an effective solution to collaboratively train the model while preserving each client's privac…
Federated LearningTumor SegmentationFederated Domain Generalization with Latent Space Inversion
Federated domain generalization (FedDG) addresses distribution shifts among clients in a federated learning framework. FedDG methods aggregate the parameters of locally trained client models to form a global model that g…
Domain GeneralizationFederated Learning