Rethinking the Representation in Federated Unsupervised Learning with Non-IID Data
Federated learning achieves effective performance in modeling decentralized data. In practice, client data are not well-labeled, which makes it potential for federated unsupervised learning (FUSL) with non-IID data. However, the performance of existing FUSL methods suffers from insufficient representations, i.e., (1) representation collapse entanglement among local and global models, and (2) inconsistent representation spaces among local models. The former indicates that representation collapse in local model will subsequently impact the global model and other local models. The latter means that clients model data representation with inconsistent parameters due to the deficiency of supervision signals. In this work, we propose FedU2 which enhances generating uniform and unified representation in FUSL with non-IID data. Specifically, FedU2 consists of flexible uniform regularizer (FUR) and efficient unified aggregator (EUA). FUR in each client avoids representation collapse via dispersing samples uniformly, and EUA in server promotes unified representation by constraining consistent client model updating. To extensively validate the performance of FedU2, we conduct both cross-device and cross-silo evaluation experiments on two benchmark datasets, i.e., CIFAR10 and CIFAR100.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningFederated Unsupervised LearningSimilar Papers 제목 키워드 기반
FedX: Unsupervised Federated Learning with Cross Knowledge Distillation
This paper presents FedX, an unsupervised federated learning framework. Our model learns unbiased representation from decentralized and heterogeneous local data. It employs a two-sided knowledge distillation with contras…
Contrastive LearningFederated LearningKnowledge DistillationFedRIR: Rethinking Information Representation in Federated Learning
Mobile and Web-of-Things (WoT) devices at the network edge generate vast amounts of data for machine learning applications, yet privacy concerns hinder centralized model training. Federated Learning (FL) allows clients (…
Federated LearningFederated Clustering: An Unsupervised Cluster-Wise Training for Decentralized Data Distributions
Federated Learning (FL) is a pivotal approach in decentralized machine learning, especially when data privacy is crucial and direct data sharing is impractical. While FL is typically associated with supervised learning, …
ClusteringFederated LearningFederated Unsupervised Representation Learning
To leverage enormous unlabeled data on distributed edge devices, we formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model withou…
Federated LearningRepresentation LearningFORLA:Federated Object-centric Representation Learning with Slot Attention
Learning efficient visual representations across heterogeneous unlabeled datasets remains a central challenge in federated learning. Effective federated representations require features that are jointly informative acros…
DecoderFederated LearningObject DiscoveryRepresentation Learning