Deep Probabilistic Ensembles: Approximate Variational Inference through KL Regularization
In this paper, we introduce Deep Probabilistic Ensembles (DPEs), a scalable technique that uses a regularized ensemble to approximate a deep Bayesian Neural Network (BNN). We do so by incorporating a KL divergence penalty term into the training objective of an ensemble, derived from the evidence lower bound used in variational inference. We evaluate the uncertainty estimates obtained from our models for active learning on visual classification. Our approach steadily improves upon active learning baselines as the annotation budget is increased.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningGeneral ClassificationVariational InferenceSimilar Papers 제목 키워드 기반
Probabilistic Deep Learning with Generalised Variational Inference
We study probabilistic Deep Learning methods through the lens of Approximate Bayesian Inference. In particular, we examine Bayesian Neural Networks (BNNs), which usually suffer from multiple ill-posed assumptions such as…
Bayesian InferenceDeep LearningProbabilistic Deep LearningUncertainty Quantification+1Bayesian posterior approximation with stochastic ensembles
We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochastic ensembles are formulated as families of distrib…
Bayesian Inferenceimage-classificationImage ClassificationVariational InferenceRefining the variational posterior through iterative optimization
Variational inference (VI) is a popular approach for approximate Bayesian inference that is particularly promising for highly parameterized models such as deep neural networks. A key challenge of variational inference i…
Bayesian InferenceVariational InferenceVariational Inference MPC for Bayesian Model-based Reinforcement Learning
In recent studies on model-based reinforcement learning (MBRL), incorporating uncertainty in forward dynamics is a state-of-the-art strategy to enhance learning performance, making MBRLs competitive to cutting-edge model…
Bayesian InferenceModel-based Reinforcement LearningModel Predictive Controlreinforcement-learning+4Wasserstein Variational Inference
This paper introduces Wasserstein variational inference, a new form of approximate Bayesian inference based on optimal transport theory. Wasserstein variational inference uses a new family of divergences that includes bo…
Bayesian InferenceVariational Inference