Accurate Uncertainty Estimation and Decomposition in Ensemble Learning
Ensemble learning is a standard approach to building machine learning systems that capture complex phenomena in real-world data. An important aspect of these systems is the complete and valid quantification of model uncertainty. We introduce a Bayesian nonparametric ensemble (BNE) approach that augments an existing ensemble model to account for different sources of model uncertainty. BNE augments a model's prediction and distribution functions using Bayesian nonparametric machinery. It has a theoretical guarantee in that it robustly estimates the uncertainty patterns in the data distribution, and can decompose its overall predictive uncertainty into distinct components that are due to different sources of noise and error. We show that our method achieves accurate uncertainty estimates under complex observational noise, and illustrate its real-world utility in terms of uncertainty decomposition and model bias detection for an ensemble in predict air pollution exposures in Eastern Massachusetts, USA.
Code (0)
등록된 구현이 없습니다.
Tasks
Bias DetectionEnsemble LearningvalidSimilar Papers 제목 키워드 기반
A general framework for ensemble distribution distillation
Ensembles of neural networks have been shown to give better performance than single networks, both in terms of predictions and uncertainty estimation. Additionally, ensembles allow the uncertainty to be decomposed into a…
regressionVariance-Gated Ensembles: An Epistemic-Aware Framework for Uncertainty Estimation
Machine learning applications require fast and reliable per-sample uncertainty estimation. A common approach is to use predictive distributions from Bayesian or approximation methods and additively decompose uncertainty …
Introduction and Exemplars of Uncertainty Decomposition
Uncertainty plays a crucial role in the machine learning field. Both model trustworthiness and performance require the understanding of uncertainty, especially for models used in high-stake applications where errors can …
Autonomous DrivingEnsemble LearningGaussian ProcessesMedical DiagnosisUncertainty Estimation using Variance-Gated Distributions
Evaluation of per-sample uncertainty quantification from neural networks is essential for decision-making involving high-risk applications. A common approach is to use the predictive distribution from Bayesian or approxi…
Quantum parameter estimation with uncertainty quantification from continuous measurement data using neural network ensembles
We show that ensembles of deep neural networks, called deep ensembles, can be used to perform quantum parameter estimation while also providing a means for quantifying uncertainty in parameter estimates, which is a key a…
Bayesian Inference