Efficient Gradient-Free Variational Inference using Policy Search
Inference from complex distributions is a common problem in machine learning needed for many Bayesian methods. We propose an efficient, gradient-free method for learning general GMM approximations of multimodal distributions based on recent insights from stochastic search methods. Our method establishes information-geometric trust regions to ensure efficient exploration of the sampling space and stability of the GMM updates, allowing for efficient estimation of multi-variate Gaussian variational distributions. For GMMs, we apply a variational lower bound to decompose the learning objective into sub-problems given by learning the individual mixture components and the coefficients. The number of mixture components is adapted online in order to allow for arbitrary exact approximations. We demonstrate on several domains that we can learn significantly better approximations than competing variational inference methods and that the quality of samples drawn from our approximations is on par with samples created by state-of-the-art MCMC samplers that require significantly more computational resources.
Code (1)
Tasks
Efficient ExplorationVariational InferenceSimilar Papers 제목 키워드 기반
Variational Inference for Policy Gradient
Inspired by the seminal work on Stein Variational Inference and Stein Variational Policy Gradient, we derived a method to generate samples from the posterior variational parameter distribution by \textit{explicitly} mini…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Variational InferenceDeep Active Inference as Variational Policy Gradients
Active Inference is a theory of action arising from neuroscience which casts action and planning as a bayesian inference problem to be solved by minimizing a single quantity - the variational free energy. Active Inferenc…
Bayesian Inferencereinforcement-learningReinforcement LearningReinforcement Learning (RL)Policy Gradient Approach to Compilation of Variational Quantum Circuits
We propose a method for finding approximate compilations of quantum unitary transformations, based on techniques from policy gradient reinforcement learning. The choice of a stochastic policy allows us to rephrase the op…
Stein Variational Policy Gradient
Policy gradient methods have been successfully applied to many complex reinforcement learning problems. However, policy gradient methods suffer from high variance, slow convergence, and inefficient exploration. In this w…
Bayesian Inferencecontinuous-controlContinuous ControlPolicy Gradient Methods+3Control as Hybrid Inference
The field of reinforcement learning can be split into model-based and model-free methods. Here, we unify these approaches by casting model-free policy optimisation as amortised variational inference, and model-based plan…
continuous-controlContinuous ControlVariational Inference