Model-Contrastive Federated Learning
Federated learning enables multiple parties to collaboratively train a machine learning model without communicating their local data. A key challenge in federated learning is to handle the heterogeneity of local data distribution across parties. Although many studies have been proposed to address this challenge, we find that they fail to achieve high performance in image datasets with deep learning models. In this paper, we propose MOON: model-contrastive federated learning. MOON is a simple and effective federated learning framework. The key idea of MOON is to utilize the similarity between model representations to correct the local training of individual parties, i.e., conducting contrastive learning in model-level. Our extensive experiments show that MOON significantly outperforms the other state-of-the-art federated learning algorithms on various image classification tasks.
Code (6)
Tasks
Contrastive LearningFederated Learningimage-classificationImage ClassificationmodelMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Decoupled Contrastive Learning for Federated Learning
Federated learning is a distributed machine learning paradigm that allows multiple participants to train a shared model by exchanging model updates instead of their raw data. However, its performance is degraded compared…
Contrastive LearningFederated LearningCCFC: Bridging Federated Clustering and Contrastive Learning
Federated clustering, an essential extension of centralized clustering for federated scenarios, enables multiple data-holding clients to collaboratively group data while keeping their data locally. In centralized scenari…
ClusteringContrastive LearningRepresentation LearningRelaxed Contrastive Learning for Federated Learning
We propose a novel contrastive learning framework to effectively address the challenges of data heterogeneity in federated learning. We first analyze the inconsistency of gradient updates across clients during local trai…
Contrastive LearningFederated LearningContrastive Federated Learning with Tabular Data Silos
Learning from data silos is a difficult task for organizations that need to obtain knowledge of objects that appeared in multiple independent data silos. Objects in multi-organizations, such as government agents, are ref…
Contrastive LearningFederated LearningSelf-LearningBreaking the Prototype Bias Loop: Confidence-Aware Federated Contrastive Learning for Highly Imbalanced Clients
Local class imbalance and data heterogeneity across clients often trap prototype-based federated contrastive learning in a prototype bias loop: biased local prototypes induced by imbalanced data are aggregated into biase…
Contrastive Learning