Federated Multi-Mini-Batch: An Efficient Training Approach to Federated Learning in Non-IID Environments
Federated learning has faced performance and network communication challenges, especially in the environments where the data is not independent and identically distributed (IID) across the clients. To address the former challenge, we introduce the federated-centralized concordance property and show that the federated single-mini-batch training approach can achieve comparable performance as the corresponding centralized training in the Non-IID environments. To deal with the latter, we present the federated multi-mini-batch approach and illustrate that it can establish a trade-off between the performance and communication efficiency and outperforms federated averaging in the Non-IID settings.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
On Large-Cohort Training for Federated Learning
Federated learning methods typically learn a model by iteratively sampling updates from a population of clients. In this work, we explore how the number of clients sampled at each round (the cohort size) impacts the qual…
FairnessFederated LearningFederated Motor Imagery Classification for Privacy-Preserving Brain-Computer Interfaces
Training an accurate classifier for EEG-based brain-computer interface (BCI) requires EEG data from a large number of users, whereas protecting their data privacy is a critical consideration. Federated learning (FL) is a…
Brain Computer InterfaceEEGFederated LearningMotor Imagery+1Implicit Gradient Alignment in Distributed and Federated Learning
A major obstacle to achieving global convergence in distributed and federated learning is the misalignment of gradients across clients, or mini-batches due to heterogeneity and stochasticity of the distributed data. In t…
Federated LearningOptimal Batch Allocation for Wireless Federated Learning
Federated learning aims to construct a global model that fits the dataset distributed across local devices without direct access to private data, leveraging communication between a server and the local devices. In the co…
Federated LearningHybrid Batch Normalisation: Resolving the Dilemma of Batch Normalisation in Federated Learning
Batch Normalisation (BN) is widely used in conventional deep neural network training to harmonise the input-output distributions for each batch of data. However, federated learning, a distributed learning paradigm, faces…
Federated Learning