paper-with-me

홈 › Papers

Graph Positional Encoding via Random Feature Propagation

2023-03-06 · Moshe Eliasof, Fabrizio Frasca, Beatrice Bevilacqua, Eran Treister, Gal Chechik, Haggai Maron

Two main families of node feature augmentation schemes have been explored for enhancing GNNs: random features and spectral positional encoding. Surprisingly, however, there is still no clear understanding of the relation between these two augmentation schemes. Here we propose a novel family of positional encoding schemes which draws a link between the above two approaches and improves over both. The new approach, named Random Feature Propagation (RFP), is inspired by the power iteration method and its generalizations. It concatenates several intermediate steps of an iterative algorithm for computing the dominant eigenvectors of a propagation matrix, starting from random node features. Notably, these propagation steps are based on graph-dependent propagation operators that can be either predefined or learned. We explore the theoretical and empirical benefits of RFP. First, we provide theoretical justifications for using random features, for incorporating early propagation steps, and for using multiple random initializations. Then, we empirically demonstrate that RFP significantly outperforms both spectral PE and random features in multiple node classification and graph classification benchmarks.

📄 PDF Abstract BibTeX arXiv:2303.02918

Code (0)

등록된 구현이 없습니다.

Tasks

Graph ClassificationNode Classification

Methods 이 논문이 사용한 방법론

1x1 Convolution A 1 x 1 Convolution is a convolution with some special properties in that it can be used for dimensionality reduction,…
Sigmoid Activation 설명 없음
RFP 설명 없음

Similar Papers 제목 키워드 기반

Homomorphism Counts as Structural Encodings for Graph Learning

2024-10-24 · Linus Bao, Emily Jin, Michael Bronstein, İsmail İlkan Ceylan 외

Graph Transformers are popular neural networks that extend the well-known Transformer architecture to the graph domain. These architectures operate by applying self-attention on graph nodes and incorporating graph struct…

Graph LearningMolecular Property PredictionProperty Prediction

Hierarchical Opacity Propagation for Image Matting

2020-04-07 · Yaoyi Li, Qingyao Xu, Hongtao Lu

Natural image matting is a fundamental problem in computational photography and computer vision. Deep neural networks have seen the surge of successful methods in natural image matting in recent years. In contrast to tra…

Image Matting

Equivariant and Stable Positional Encoding for More Powerful Graph Neural Networks

2022-03-01 · ICLR 2022 4 · Haorui Wang, Haoteng Yin, Muhan Zhang, Pan Li

Graph neural networks (GNN) have shown great advantages in many graph-based learning tasks but often fail to predict accurately for a task-based on sets of nodes such as link/motif prediction and so on. Many works have r…

Link PredictionPrediction

Self-Attention in Colors: Another Take on Encoding Graph Structure in Transformers

2023-04-21 · Romain Menegaux, Emmanuel Jehanno, Margot Selosse, Julien Mairal

We introduce a novel self-attention mechanism, which we call CSA (Chromatic Self-Attention), which extends the notion of attention scores to attention _filters_, independently modulating the feature channels. We showcase…

Graph Regression

Comparing Graph Transformers via Positional Encodings

2024-02-22 · Mitchell Black, Zhengchao Wan, Gal Mishne, Amir Nayyeri 외

The distinguishing power of graph transformers is closely tied to the choice of positional encoding: features used to augment the base transformer with information about the graph. There are two primary types of position…

Navigate