Gradient Gating for Deep Multi-Rate Learning on Graphs

2022-10-02 · T. Konstantin Rusch, Benjamin P. Chamberlain, Michael W. Mahoney, Michael M. Bronstein, Siddhartha Mishra

We present Gradient Gating (G$^2$), a novel framework for improving the performance of Graph Neural Networks (GNNs). Our framework is based on gating the output of GNN layers with a mechanism for multi-rate flow of message passing information across nodes of the underlying graph. Local gradients are harnessed to further modulate message passing updates. Our framework flexibly allows one to use any basic GNN layer as a wrapper around which the multi-rate gradient gating mechanism is built. We rigorously prove that G$^2$ alleviates the oversmoothing problem and allows the design of deep GNNs. Empirical results are presented to demonstrate that the proposed framework achieves state-of-the-art performance on a variety of graph learning tasks, including on large-scale heterophilic graphs.

📄 PDF Abstract BibTeX arXiv:2210.00513

Code (1)

tk-rusch/gradientgating 공식 구현 pytorch

Tasks

Graph Learning Node Classification

Learning Graph Weighted Models on Pictures

2018-06-21 · Philip Amortila, Guillaume Rabusseau

Graph Weighted Models (GWMs) have recently been proposed as a natural generalization of weighted automata over strings and trees to arbitrary families of labeled graphs (and hypergraphs). A GWM generically associates a l…

Mitigating Modality Collapse in Multimodal VAEs via Impartial Optimization

2022-06-09 · Adrián Javaloy, Maryam Meghdadi, Isabel Valera

A number of variational autoencoders (VAEs) have recently emerged with the aim of modeling multimodal data, e.g., to jointly model images and their corresponding captions. Still, multimodal VAEs tend to focus solely on a…

Sporadic Gradient Tracking over Directed Graphs: A Theoretical Perspective on Decentralized Federated Learning

2026-01-31 · Shahryar Zehtabi, Dong-Jun Han, Seyyedali Hosseinalipour, Christopher Brinton arxiv

Decentralized Federated Learning (DFL) enables clients with local data to collaborate in a peer-to-peer manner to train a generalized model. In this paper, we unify two branches of work that have separately solved import…

Image Classification Federated Learning

Backprop-Q: Generalized Backpropagation for Stochastic Computation Graphs

2018-07-25 · Xiaoran Xu, Songpeng Zu, Yuan Zhang, Hanning Zhou 외

In real-world scenarios, it is appealing to learn a model carrying out stochastic operations internally, known as stochastic computation graphs (SCGs), rather than learning a deterministic mapping. However, standard back…

Reinforcement Learning Reinforcement Learning (RL)

Decoupled Neural Interfaces using Synthetic Gradients

2016-08-18 · ICML 2017 8 · Max Jaderberg, Wojciech Marian Czarnecki, Simon Osindero, Oriol Vinyals 외

Training directed neural networks typically requires forward-propagating data through a computation graph, followed by backpropagating error signal, to produce weight updates. All layers, or more generally, modules, of t…