Federated Graph Representation Learning using Self-Supervision
Federated graph representation learning (FedGRL) brings the benefits of distributed training to graph structured data while simultaneously addressing some privacy and compliance concerns related to data curation. However, several interesting real-world graph data characteristics viz. label deficiency and downstream task heterogeneity are not taken into consideration in current FedGRL setups. In this paper, we consider a realistic and novel problem setting, wherein cross-silo clients have access to vast amounts of unlabeled data with limited or no labeled data and additionally have diverse downstream class label domains. We then propose a novel FedGRL formulation based on model interpolation where we aim to learn a shared global model that is optimized collaboratively using a self-supervised objective and gets downstream task supervision through local client models. We provide a specific instantiation of our general formulation using BGRL a SoTA self-supervised graph representation learning method and we empirically verify its effectiveness through realistic cross-slio datasets: (1) we adapt the Twitch Gamer Network which naturally simulates a cross-geo scenario and show that our formulation can provide consistent and avg. 6.1% gains over traditional supervised federated learning objectives and on avg. 1.7% gains compared to individual client specific self-supervised training and (2) we construct and introduce a new cross-silo dataset called Amazon Co-purchase Networks that have both the characteristics of the motivated problem setting. And, we witness on avg. 11.5% gains over traditional supervised federated learning and on avg. 1.9% gains over individually trained self-supervised models. Both experimental results point to the effectiveness of our proposed formulation. Finally, both our novel problem setting and dataset contributions provide new avenues for the research in FedGRL.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningGraph Representation LearningRepresentation LearningSimilar Papers 제목 키워드 기반
FedGL: Federated Graph Learning Framework with Global Self-Supervision
Graph data are ubiquitous in the real world. Graph learning (GL) tries to mine and analyze graph data so that valuable information can be discovered. Existing GL methods are designed for centralized scenarios. However, i…
Federated LearningGraph LearningPrivacy PreservingPseudo LabelPerFedRec++: Enhancing Personalized Federated Recommendation with Self-Supervised Pre-Training
Federated recommendation systems employ federated learning techniques to safeguard user privacy by transmitting model parameters instead of raw user data between user devices and the central server. Nevertheless, the cur…
Federated LearningGraph LearningGraph Neural NetworkPrivacy Preserving+2When Does Self-Supervision Help Graph Convolutional Networks?
Self-supervision as an emerging technique has been employed to train convolutional neural networks (CNNs) for more transferrable, generalizable, and robust representation learning of images. Its introduction to graph con…
Multi-Task LearningRepresentation LearningSelf-Supervised LearningThere is more to graphs than meets the eye: Learning universal features with self-supervision
We study the problem of learning features through self-supervision that are generalisable to multiple graphs. State-of-the-art graph self-supervision restricts training to only one graph, resulting in graph-specific mode…
Node ClassificationRepresentation LearningSelf-Supervised LearningFederated Self-Supervised Learning for Automatic Modulation Classification under Non-IID and Class-Imbalanced Data
Training automatic modulation classification (AMC) models on centrally aggregated data raises privacy concerns, incurs communication overhead, and often fails to confer robustness to channel shifts. Federated learning (F…
Self-Supervised LearningRepresentation LearningFederated Learning