SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks
Graph Neural Networks (GNNs) are the first choice methods for graph machine learning problems thanks to their ability to learn state-of-the-art level representations from graph-structured data. However, centralizing a massive amount of real-world graph data for GNN training is prohibitive due to user-side privacy concerns, regulation restrictions, and commercial competition. Federated Learning is the de-facto standard for collaborative training of machine learning models over many distributed edge devices without the need for centralization. Nevertheless, training graph neural networks in a federated setting is vaguely defined and brings statistical and systems challenges. This work proposes SpreadGNN, a novel multi-task federated training framework capable of operating in the presence of partial labels and absence of a central server for the first time in the literature. SpreadGNN extends federated multi-task learning to realistic serverless settings for GNNs, and utilizes a novel optimization algorithm with a convergence guarantee, Decentralized Periodic Averaging SGD (DPA-SGD), to solve decentralized multi-task learning problems. We empirically demonstrate the efficacy of our framework on a variety of non-I.I.D. distributed graph-level molecular property prediction datasets with partial labels. Our results show that SpreadGNN outperforms GNN models trained over a central server-dependent federated learning system, even in constrained topologies. The source code is publicly available at https://github.com/FedML-AI/SpreadGNN
Code (1)
Tasks
BIG-bench Machine LearningFederated LearningMolecular Property PredictionMulti-Task LearningProperty PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Training Heterogeneous Client Models using Knowledge Distillation in Serverless Federated Learning
Federated Learning (FL) is an emerging machine learning paradigm that enables the collaborative training of a shared global model across distributed clients while keeping the data decentralized. Recent works on designing…
Federated LearningKnowledge DistillationServerless Federated AUPRC Optimization for Multi-Party Collaborative Imbalanced Data Mining
Multi-party collaborative training, such as distributed learning and federated learning, is used to address the big data challenges. However, traditional multi-party collaborative training algorithms were mainly designed…
Federated LearningStochastic OptimizationTowards Serverless Semi-Decentralized Federated Learning with Heterogeneous Optimizers
We investigate cluster formation, involving the number and composition of clusters, in decentralized federated learning (FL) with heterogeneous machine learning (ML) optimizers. While clustering in centralized FL has ena…
Federated LearningFedLesScan: Mitigating Stragglers in Serverless Federated Learning
Federated Learning (FL) is a machine learning paradigm that enables the training of a shared global model across distributed clients while keeping the training data local. While most prior work on designing systems for F…
Federated LearningServerless Federated Learning with flwr-serverless
Federated learning is becoming increasingly relevant and popular as we witness a surge in data collection and storage of personally identifiable information. Alongside these developments there have been many proposals fr…
Federated Learning