Federated Unsupervised Representation Learning
To leverage enormous unlabeled data on distributed edge devices, we formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision while preserving data privacy. FURL poses two new challenges: (1) data distribution shift (Non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces. (2) without the unified information among clients in FURL, the representations across clients would be misaligned. To address these challenges, we propose Federated Constrastive Averaging with dictionary and alignment (FedCA) algorithm. FedCA is composed of two key modules: (1) dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and (2) alignment module to align the representation of each client on a base model trained on a public data. We adopt the contrastive loss for local model training. Through extensive experiments with three evaluation protocols in IID and Non-IID settings, we demonstrate that FedCA outperforms all baselines with significant margins.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningRepresentation LearningSimilar Papers 제목 키워드 기반
FedX: Unsupervised Federated Learning with Cross Knowledge Distillation
This paper presents FedX, an unsupervised federated learning framework. Our model learns unbiased representation from decentralized and heterogeneous local data. It employs a two-sided knowledge distillation with contras…
Contrastive LearningFederated LearningKnowledge DistillationFederated Clustering: An Unsupervised Cluster-Wise Training for Decentralized Data Distributions
Federated Learning (FL) is a pivotal approach in decentralized machine learning, especially when data privacy is crucial and direct data sharing is impractical. While FL is typically associated with supervised learning, …
ClusteringFederated LearningFORLA:Federated Object-centric Representation Learning with Slot Attention
Learning efficient visual representations across heterogeneous unlabeled datasets remains a central challenge in federated learning. Effective federated representations require features that are jointly informative acros…
DecoderFederated LearningObject DiscoveryRepresentation LearningJigsaw Game: Federated Clustering
Federated learning has recently garnered significant attention, especially within the domain of supervised learning. However, despite the abundance of unlabeled data on end-users, unsupervised learning problems such as c…
ClusteringFederated LearningRepresentation LearningPractical Vertical Federated Learning with Unsupervised Representation Learning
As societal concerns on data privacy recently increase, we have witnessed data silos among multiple parties in various applications. Federated learning emerges as a new learning paradigm that enables multiple parties to …
Federated LearningPrivacy PreservingRepresentation LearningVertical Federated Learning