A Self-Attention Network based Node Embedding Model
Despite several signs of progress have been made recently, limited research has been conducted for an inductive setting where embeddings are required for newly unseen nodes -- a setting encountered commonly in practical applications of deep learning for graph networks. This significantly affects the performances of downstream tasks such as node classification, link prediction or community extraction. To this end, we propose SANNE -- a novel unsupervised embedding model -- whose central idea is to employ a transformer self-attention network to iteratively aggregate vector representations of nodes in random walks. Our SANNE aims to produce plausible embeddings not only for present nodes, but also for newly unseen nodes. Experimental results show that the proposed SANNE obtains state-of-the-art results for the node classification task on well-known benchmark datasets.
Code (1)
Tasks
General ClassificationLink PredictionmodelNode ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Hierarchical Attention Models for Multi-Relational Graphs
We present Bi-Level Attention-Based Relational Graph Convolutional Networks (BR-GCN), unique neural network architectures that utilize masked self-attentional layers with relational graph convolutions, to effectively ope…
Graph AttentionLink PredictionNode ClassificationRelationUnsupervised Universal Self-Attention Network for Graph Classification
Existing graph embedding models often have weaknesses in exploiting graph structure similarities, potential dependencies among nodes and global network properties. To this end, we present U2GAN, a novel unsupervised mode…
ClassificationGraph ClassificationGraph EmbeddingPay Attention to Relations: Multi-embeddings for Attributed Multiplex Networks
Graph Convolutional Neural Networks (GCNs) have become effective machine learning algorithms for many downstream network mining tasks such as node classification, link prediction, and community detection. However, most G…
Community DetectionLink PredictionNetwork EmbeddingNode ClassificationDifferential Encoding for Improved Representation Learning over Graphs
Combining the message-passing paradigm with the global attention mechanism has emerged as an effective framework for learning over graphs. The message-passing paradigm and the global attention mechanism fundamentally gen…
Graph Representation LearningRepresentation LearningHDMI: High-order Deep Multiplex Infomax
Networks have been widely used to represent the relations between objects such as academic networks and social networks, and learning embedding for networks has thus garnered plenty of research attention. Self-supervised…
Node ClassificationRepresentation LearningVocal Bursts Intensity Prediction