Challenges in Markov chain Monte Carlo for Bayesian neural networks
Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges in sampling from the parameter posterior of a neural network via MCMC. Such challenges culminate to lack of convergence to the parameter posterior. Nevertheless, this paper shows that a non-converged Markov chain, generated via MCMC sampling from the parameter space of a neural network, can yield via Bayesian marginalization a valuable posterior predictive distribution of the output of the neural network. Classification examples based on multilayer perceptrons showcase highly accurate posterior predictive distributions. The postulate of limited scope for MCMC developments in BNNs is partially valid; an asymptotically exact parameter posterior seems less plausible, yet an accurate posterior predictive distribution is a tenable research avenue.
Code (1)
Tasks
Bayesian InferencevalidSimilar Papers 제목 키워드 기반
Multilevel Gibbs Sampling for Bayesian Regression
Bayesian regression remains a simple but effective tool based on Bayesian inference techniques. For large-scale applications, with complicated posterior distributions, Markov Chain Monte Carlo methods are applied. To imp…
Bayesian InferenceClusteringregressionScalable Importance Tempering and Bayesian Variable Selection
We propose a Monte Carlo algorithm to sample from high dimensional probability distributions that combines Markov chain Monte Carlo and importance sampling. We provide a careful theoretical analysis, including guarantees…
Variable SelectionAn Annealed Sequential Monte Carlo Method for Bayesian Phylogenetics
We describe an "embarrassingly parallel" method for Bayesian phylogenetic inference, annealed Sequential Monte Carlo, based on recent advances in the Sequential Monte Carlo literature such as adaptive determination of an…
Stochastic gradient Markov chain Monte Carlo
Markov chain Monte Carlo (MCMC) algorithms are generally regarded as the gold standard technique for Bayesian inference. They are theoretically well-understood and conceptually simple to apply in practice. The drawback o…
Bayesian InferenceDifferentially Private Markov Chain Monte Carlo
Recent developments in differentially private (DP) machine learning and DP Bayesian learning have enabled learning under strong privacy guarantees for the training data subjects. In this paper, we further extend the appl…