Federated Learning on Heterogeneous and Long-Tailed Data via Classifier Re-Training with Federated Features
Federated learning (FL) provides a privacy-preserving solution for distributed machine learning tasks. One challenging problem that severely damages the performance of FL models is the co-occurrence of data heterogeneity and long-tail distribution, which frequently appears in real FL applications. In this paper, we reveal an intriguing fact that the biased classifier is the primary factor leading to the poor performance of the global model. Motivated by the above finding, we propose a novel and privacy-preserving FL method for heterogeneous and long-tailed data via Classifier Re-training with Federated Features (CReFF). The classifier re-trained on federated features can produce comparable performance as the one re-trained on real data in a privacy-preserving manner without information leakage of local data or class distribution. Experiments on several benchmark datasets show that the proposed CReFF is an effective solution to obtain a promising FL model under heterogeneous and long-tailed data. Comparative results with the state-of-the-art FL methods also validate the superiority of CReFF. Our code is available at https://github.com/shangxinyi/CReFF-FL.
Code (2)
Tasks
Federated LearningPrivacy PreservingSimilar Papers 제목 키워드 기반
Decoupled Federated Learning on Long-Tailed and Non-IID data with Feature Statistics
Federated learning is designed to enhance data security and privacy, but faces challenges when dealing with heterogeneous data in long-tailed and non-IID distributions. This paper explores an overlooked scenario where ta…
Federated LearningIntegrating Local Real Data with Global Gradient Prototypes for Classifier Re-Balancing in Federated Long-Tailed Learning
Federated Learning (FL) has become a popular distributed learning paradigm that involves multiple clients training a global model collaboratively in a data privacy-preserving manner. However, the data samples usually fol…
Federated LearningPrivacy PreservingCLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data
Federated learning (FL) provides a decentralized machine learning paradigm where a server collaborates with a group of clients to learn a global model without accessing the clients' data. User heterogeneity is a signific…
Contrastive LearningFederated Learningimage-classificationImage Classification+3FCA: Taming Long-tailed Federated Medical Image Classification by Classifier Anchoring
Limited training data and severe class imbalance impose significant challenges to developing clinically robust deep learning models. Federated learning (FL) addresses the former by enabling different medical clients to c…
Federated Learningimage-classificationImage ClassificationLesion Classification+2Fed-GraB: Federated Long-tailed Learning with Self-Adjusting Gradient Balancer
Data privacy and long-tailed distribution are the norms rather than the exception in many real-world tasks. This paper investigates a federated long-tailed learning (Fed-LT) task in which each client holds a locally hete…