paper-with-me

홈 › Papers

Self-supervised edge features for improved Graph Neural Network training

2020-06-23 · Arijit Sehanobish, Neal G. Ravindra, David van Dijk

Graph Neural Networks (GNN) have been extensively used to extract meaningful representations from graph structured data and to perform predictive tasks such as node classification and link prediction. In recent years, there has been a lot of work incorporating edge features along with node features for prediction tasks. One of the main difficulties in using edge features is that they are often handcrafted, hard to get, specific to a particular domain, and may contain redundant information. In this work, we present a framework for creating new edge features, applicable to any domain, via a combination of self-supervised and unsupervised learning. In addition to this, we use Forman-Ricci curvature as an additional edge feature to encapsulate the local geometry of the graph. We then encode our edge features via a Set Transformer and combine them with node features extracted from popular GNN architectures for node classification in an end-to-end training scheme. We validate our work on three biological datasets comprising of single-cell RNA sequencing data of neurological disease, \textit{in vitro} SARS-CoV-2 infection, and human COVID-19 patients. We demonstrate that our method achieves better performance on node classification tasks over baseline Graph Attention Network (GAT) and Graph Convolutional Network (GCN) models. Furthermore, given the attention mechanism on edge and node features, we are able to interpret the cell types and genes that determine the course and severity of COVID-19, contributing to a growing list of potential disease biomarkers and therapeutic targets.

📄 PDF Abstract BibTeX arXiv:2007.04777

Code (1)

nealgravindra/self-supervsed_edge_feats pytorch

Tasks

General ClassificationGraph AttentionGraph Neural NetworkLink PredictionNode Classification

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Position-Wise Feed-Forward Layer 설명 없음
Set Transformer Many machine learning tasks such as multiple instance learning, 3D shape recognition, and few-shot image classification are defined on sets of instances. Since solutions to such…
Residual Connection 설명 없음
Attention 설명 없음
Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Similar Papers 제목 키워드 기반

Toward Improved Generalization: Meta Transfer of Self-supervised Knowledge on Graphs

2022-12-16 · Wenhui Cui, Haleh Akrami, Anand A. Joshi, Richard M. Leahy

Despite the remarkable success achieved by graph convolutional networks for functional brain activity analysis, the heterogeneity of functional patterns and the scarcity of imaging data still pose challenges in many task…

Meta-LearningRepresentation LearningSelf-Supervised LearningTransfer Learning

Multi-task Self-distillation for Graph-based Semi-Supervised Learning

2021-12-02 · Yating Ren, Junzhong Ji, Lingfeng Niu, Minglong Lei

Graph convolutional networks have made great progress in graph-based semi-supervised learning. Existing methods mainly assume that nodes connected by graph edges are prone to have similar attributes and labels, so that t…

Node Classification

Self-Supervised Relation Alignment for Scene Graph Generation

2023-02-02 · Bicheng Xu, Renjie Liao, Leonid Sigal

The goal of scene graph generation is to predict a graph from an input image, where nodes correspond to identified and localized objects and edges to their corresponding interaction predicates. Existing methods are train…

Graph GenerationRelationRelation PredictionScene Graph Generation

How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

2022-04-11 · ICLR 2021 1 · Dongkwan Kim, Alice Oh

Attention mechanism in graph neural networks is designed to assign larger weights to important neighbor nodes for better representation. However, what graph attention learns is not understood well, particularly when grap…

Graph Attention

Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut

2022-02-23 · CVPR 2022 1 · Yangtao Wang, Xi Shen, Shell Hu, Yuan Yuan 외

Transformers trained with self-supervised learning using self-distillation loss (DINO) have been shown to produce attention maps that highlight salient foreground objects. In this paper, we demonstrate a graph-based appr…

Objectobject-detectionObject DetectionObject Discovery+5