Federated Generalized Bayesian Learning via Distributed Stein Variational Gradient Descent
This paper introduces Distributed Stein Variational Gradient Descent (DSVGD), a non-parametric generalized Bayesian inference framework for federated learning. DSVGD maintains a number of non-random and interacting particles at a central server to represent the current iterate of the model global posterior. The particles are iteratively downloaded and updated by one of the agents with the end goal of minimizing the global free energy. By varying the number of particles, DSVGD enables a flexible trade-off between per-iteration communication load and number of communication rounds. DSVGD is shown to compare favorably to benchmark frequentist and Bayesian federated learning strategies, also scheduling a single device per iteration, in terms of accuracy and scalability with respect to the number of agents, while also providing well-calibrated, and hence trustworthy, predictions.
Code (1)
Tasks
Bayesian InferenceFederated LearningSchedulingSimilar Papers 제목 키워드 기반
Forget-SVGD: Particle-Based Bayesian Federated Unlearning
Variational particle-based Bayesian learning methods have the advantage of not being limited by the bias affecting more conventional parametric techniques. This paper proposes to leverage the flexibility of non-parametri…
Bayesian InferenceFederated LearningClient Selection for Federated Bayesian Learning
Distributed Stein Variational Gradient Descent (DSVGD) is a non-parametric distributed learning framework for federated Bayesian learning, where multiple clients jointly train a machine learning model by communicating a …
Personalized Bayesian Federated Learning with Wasserstein Barycenter Aggregation
Personalized Bayesian federated learning (PBFL) handles non-i.i.d. client data and quantifies uncertainty by combining personalization with Bayesian inference. However, existing PBFL methods face two limitations: restric…
Bayesian InferenceFederated LearningVariational InferenceGeneralized Variational Inference in Function Spaces: Gaussian Measures meet Bayesian Deep Learning
We develop a framework for generalized variational inference in infinite-dimensional function spaces and use it to construct a method termed Gaussian Wasserstein inference (GWI). GWI leverages the Wasserstein distance be…
Gaussian ProcessesUncertainty QuantificationVariational InferenceWasserstein Gaussianization and Efficient Variational Bayes for Robust Bayesian Synthetic Likelihood
The Bayesian Synthetic Likelihood (BSL) method is a widely-used tool for likelihood-free Bayesian inference. This method assumes that some summary statistics are normally distributed, which can be incorrect in many appli…
Bayesian Inference