EP-GAN: Unsupervised Federated Learning with Expectation-Propagation Prior GAN
Generative Adversarial Networks (GANs) are overwhelming in unsupervised learning tasks due to their expressive power in modeling fine-grained data distributions. However, it is challenging for GANs to model distributions of separate non-i.i.d. data partitions as it usually adopts an over-general prior, limiting its capability in capturing the latent structure of multiple data partitions and thus leading to mode collapse. In this paper, we present a new Bayesian GAN, dubbed expectation propagation prior GAN (EP-GAN), which addresses the above challenge of modeling non-i.i.d. federated data through imposing a partition-invariant prior distribution on a Bayesian GAN. Furthermore, unlike most existing algorithms for deep-learning-based EP inference that require numerical quadrature, here we propose a closed-form solution for each update step of EP, leading to a more efficient solution for federated data modeling. Experiments on both synthetic extremely non-i.i.d. image data partitions and realistic non-i.i.d. speech recognition tasks demonstrate that our framework effectively alleviates the performance deterioration caused by non-i.i.d. data.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated Learningspeech-recognitionSpeech RecognitionSimilar Papers 제목 키워드 기반
Federated Learning as Variational Inference: A Scalable Expectation Propagation Approach
The canonical formulation of federated learning treats it as a distributed optimization problem where the model parameters are optimized against a global loss function that decomposes across client loss functions. A rece…
Distributed OptimizationFederated LearningVariational InferenceUnsupervised Multi-View Object Segmentation Using Radiance Field Propagation
We present radiance field propagation (RFP), a novel approach to segmenting objects in 3D during reconstruction given only unlabeled multi-view images of a scene. RFP is derived from emerging neural radiance field-based …
3D Object EditingNeRFObjectScene Segmentation+2An Expectation-Maximization Perspective on Federated Learning
Federated learning describes the distributed training of models across multiple clients while keeping the data private on-device. In this work, we view the server-orchestrated federated learning process as a hierarchical…
Federated LearningFederated Averaging as Expectation Maximization
Federated averaging (FedAvg), despite its simplicity, has been the main approach in training neural networks in the federated learning setting. In this work, we show that the algorithmic choices of the FedAvg algorithm c…
Federated LearningFast Scalable Image Restoration using Total Variation Priors and Expectation Propagation
This paper presents a scalable approximate Bayesian method for image restoration using total variation (TV) priors. In contrast to most optimization methods based on maximum a posteriori estimation, we use the expectatio…
Compressive SensingDenoisingImage Restoration