paper-with-me

Papers

Graph Attention for Heterogeneous Graphs with Positional Encoding

2025-04-03 · Nikhil Shivakumar Nayak

Graph Neural Networks (GNNs) have emerged as the de facto standard for modeling graph data, with attention mechanisms and transformers significantly enhancing their performance on graph-based tasks. Despite these advancements, the performance of GNNs on heterogeneous graphs often remains complex, with networks generally underperforming compared to their homogeneous counterparts. This work benchmarks various GNN architectures to identify the most effective methods for heterogeneous graphs, with a particular focus on node classification and link prediction. Our findings reveal that graph attention networks excel in these tasks. As a main contribution, we explore enhancements to these attention networks by integrating positional encodings for node embeddings. This involves utilizing the full Laplacian spectrum to accurately capture both the relative and absolute positions of each node within the graph, further enhancing performance on downstream tasks such as node classification and link prediction.

📄 PDF Abstract BibTeX arXiv:2504.02938

Code (1)

nikhilnayak-debug/am220 공식 구현 pytorch

Tasks

Graph AttentionLink PredictionNode Classification

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음
Focus 설명 없음

Similar Papers 제목 키워드 기반

Relational Graph Transformer

2025-05-16 · Vijay Prakash Dwivedi, Sri Jaladi, Yangyi Shen, Federico López 외

Relational Deep Learning (RDL) is a promising approach for building state-of-the-art predictive models on multi-table relational data by representing it as a heterogeneous temporal graph. However, commonly used Graph Neu…

Graph Neural Network

Rewiring with Positional Encodings for Graph Neural Networks

2022-01-29 · Rickard Brüel-Gabrielsson, Mikhail Yurochkin, Justin Solomon

Several recent works use positional encodings to extend the receptive fields of graph neural network (GNN) layers equipped with attention mechanisms. These techniques, however, extend receptive fields to the complete gra…

Graph Neural Network

SpecTRA: Spectral Transformer for Graph Representation Learning

2021-09-29 · Anson Bastos, Abhishek Nadgeri, Kuldeep Singh, Hiroki Kanezashi 외

Transformers have recently been applied in the more generic domain of graphs. For the same, researchers proposed various positional and structural encoding schemes to overcome the limitation of transformers in modeling t…

Graph Representation LearningRepresentation Learning

Communicability-Inspired Positional Encoding (CIPE)

2026-06-24 · Yipeng Zhang, Zhongtian Sun, Pietro Liò, Kelin Xia arxiv

Positional encodings (PEs) are essential for Transformers. Yet designing effective PEs for non-Euclidean graphs remains challenging. Such encodings should ideally induce an Attention-Compatible Geometry for self-attentio…

Attention-Driven Metapath Encoding in Heterogeneous Graphs

2024-12-30 · Calder Katyal

One of the emerging techniques in node classification in heterogeneous graphs is to restrict message aggregation to pre-defined, semantically meaningful structures called metapaths. This work is the first attempt to inco…

Node Classification