Federated-EM with heterogeneity mitigation and variance reduction
The Expectation Maximization (EM) algorithm is the default algorithm for inference in latent variable models. As in any other field of machine learning, applications of latent variable models to very large datasets make the use of advanced parallel and distributed architecture mandatory. This paper introduces FedEM, which is the first extension of the EM algorithm to the federated learning context. FedEM is a new communication efficient method, which handles partial participation of local devices, and is robust to heterogeneous distribution of the datasets. To alleviate the communication bottleneck, FedEM compresses appropriately defined complete data sufficient statistics. We also develop and analyze an extension of FedEM to further incorporate a variance reduction scheme. In all cases, we derive finite-time complexity bounds for smooth non-convex problems. Numerical results are presented to support our theoretical findings, as well as an application to federated missing values imputation for biodiversity monitoring.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningImputationMissing ValuesSimilar Papers 제목 키워드 기반
Federated Expectation Maximization with heterogeneity mitigation and variance reduction
The Expectation Maximization (EM) algorithm is the default algorithm for inference in latent variable models. As in any other field of machine learning, applications of latent variable models to very large datasets make …
Federated LearningImputationMissing ValuesDifferential Privacy Meets Federated Learning under Communication Constraints
The performance of federated learning systems is bottlenecked by communication costs and training variance. The communication overhead problem is usually addressed by three communication-reduction techniques, namely, mod…
Federated LearningModel CompressionQ-LocalAdam: Memory-Efficient Client-Side Adaptive Optimization for Edge Federated Learning
Federated learning on edge devices must cope with non-IID client data and tight memory budgets. Adaptive optimizers like Adam stabilize training under data heterogeneity but require storing full-precision momentum and va…
Federated LearningSociodynamics-inspired Adaptive Coalition and Client Selection in Federated Learning
Federated Learning (FL) enables privacy-preserving collaborative model training, yet its practical strength is often undermined by client data heterogeneity, which severely degrades model performance. This paper proposes…
Federated LearningPrivacy PreservingTowards Heterogeneous Quantum Federated Learning: Challenges and Solutions
Quantum federated learning (QFL) combines quantum computing and federated learning to enable decentralized model training while maintaining data privacy. QFL can improve computational efficiency and scalability by taking…
Computational EfficiencyFederated Learning