Bayesian Inference with Anchored Ensembles of Neural Networks, and Application to Exploration in Reinforcement Learning
The use of ensembles of neural networks (NNs) for the quantification of predictive uncertainty is widespread. However, the current justification is intuitive rather than analytical. This work proposes one minor modification to the normal ensembling methodology, which we prove allows the ensemble to perform Bayesian inference, hence converging to the corresponding Gaussian Process as both the total number of NNs, and the size of each, tend to infinity. This working paper provides early-stage results in a reinforcement learning setting, analysing the practicality of the technique for an ensemble of small, finite number. Using the uncertainty estimates produced by anchored ensembles to govern the exploration-exploitation process results in steadier, more stable learning.
Code (2)
Tasks
Bayesian Inferencereinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
SAE: Sequential Anchored Ensembles
Computing the Bayesian posterior of a neural network is a challenging task due to the high-dimensionality of the parameter space. Anchored ensembles approximate the posterior by training an ensemble of neural networks on…
Mixture Distributions for Scalable Bayesian Inference
Bayesian Neural Networks (BNNs) provides a mathematically grounded framework to quantify uncertainty. However BNNs are computationally inefficient, thus are generally not employed on complicated machine learning tasks. D…
Bayesian InferenceOn the Effectiveness of Mode Exploration in Bayesian Model Averaging for Neural Networks
Multiple techniques for producing calibrated predictive probabilities using deep neural networks in supervised learning settings have emerged that leverage approaches to ensemble diverse solutions discovered during cycli…
Bayesian InferenceVariational InferenceBayesian posterior approximation with stochastic ensembles
We introduce ensembles of stochastic neural networks to approximate the Bayesian posterior, combining stochastic methods such as dropout with deep ensembles. The stochastic ensembles are formulated as families of distrib…
Bayesian Inferenceimage-classificationImage ClassificationVariational InferenceOn the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference
While parameter-efficient fine-tuning methods like low-rank adaptation (LoRA) are standard for large language models, principled estimation of epistemic uncertainty remains challenging. Recent results in the LoRA regime …
parameter-efficient fine-tuningBayesian Inference