Federated Self-supervised Learning for Heterogeneous Clients
Federated Learning has become an important learning paradigm due to its privacy and computational benefits. As the field advances, two key challenges that still remain to be addressed are: (1) system heterogeneity - variability in the compute and/or data resources present on each client, and (2) lack of labeled data in certain federated settings. Several recent developments have tried to overcome these challenges independently. In this work, we propose a unified and systematic framework, \emph{Heterogeneous Self-supervised Federated Learning} (Hetero-SSFL) for enabling self-supervised learning with federation on heterogeneous clients. The proposed framework allows collaborative representation learning across all the clients without imposing architectural constraints or requiring presence of labeled data. The key idea in Hetero-SSFL is to let each client train its unique self-supervised model and enable the joint learning across clients by aligning the lower dimensional representations on a common dataset. The entire training procedure could be viewed as self and peer-supervised as both the local training and the alignment procedures do not require presence of any labeled data. As in conventional self-supervised learning, the obtained client models are task independent and can be used for varied end-tasks. We provide a convergence guarantee of the proposed framework for non-convex objectives in heterogeneous settings and also empirically demonstrate that our proposed approach outperforms the state of the art methods by a significant margin.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningRepresentation LearningSelf-Supervised LearningSimilar Papers 제목 키워드 기반
Local K-Similarity Constraint for Federated Learning with Label Noise
Federated learning on clients with noisy labels is a challenging problem, as such clients can infiltrate the global model, impacting the overall generalizability of the system. Existing methods proposed to handle noisy c…
Medical Image ClassificationFederated LearningFederated Self-Supervised Learning in Heterogeneous Settings: Limits of a Baseline Approach on HAR
Federated Learning is a new machine learning paradigm dealing with distributed model learning on independent devices. One of the many advantages of federated learning is that training data stay on devices (such as smartp…
Activity RecognitionFederated LearningHuman Activity RecognitionSelf-Supervised LearningRobust Asymmetric Heterogeneous Federated Learning with Corrupted Clients
This paper studies a challenging robust federated learning task with model heterogeneous and data corrupted clients, where the clients have different local model structures. Data corruption is unavoidable due to factors …
Contrastive LearningData AugmentationFederated LearningContrastive encoder pre-training-based clustered federated learning for heterogeneous data
Federated learning (FL) is a promising approach that enables distributed clients to collaboratively train a global model while preserving their data privacy. However, FL often suffers from data heterogeneity problems, wh…
ClusteringContrastive LearningFederated LearningFederated Self-Supervised Learning for Automatic Modulation Classification under Non-IID and Class-Imbalanced Data
Training automatic modulation classification (AMC) models on centrally aggregated data raises privacy concerns, incurs communication overhead, and often fails to confer robustness to channel shifts. Federated learning (F…
Self-Supervised LearningRepresentation LearningFederated Learning