Prediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles
Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estimates from an ensemble of neural networks. We propose a multi-objective loss function fusing quality measures related to prediction intervals and point estimates, and a penalty function, which enforces semantic integrity of the results and stabilizes the training process of the neural networks. The ensembled prediction intervals are aggregated as a split normal mixture accounting for possible multimodality and asymmetricity of the posterior predictive distribution, and resulting in prediction intervals that capture aleatoric and epistemic uncertainty. Our results show that both our quality-driven loss function and our aggregation method contribute to well-calibrated prediction intervals and point estimates.
Code (1)
Tasks
PredictionPrediction IntervalsSimilar Papers 제목 키워드 기반
Conditional validity of heteroskedastic conformal regression
Conformal prediction, and split conformal prediction as a specific implementation, offer a distribution-free approach to estimating prediction intervals with statistical guarantees. Recent work has shown that split confo…
Conformal PredictionPredictionPrediction IntervalsregressionSmoothing-Based Conformal Prediction for Balancing Efficiency and Interpretability
Conformal Prediction (CP) is a distribution-free framework for constructing statistically rigorous prediction sets. While popular variants such as CD-split improve CP's efficiency, they often yield prediction sets compos…
On Optimal Data Splitting for Split Conformal Prediction
Conformal prediction and its variants, including the split conformal prediction, provide a distribution-free framework for uncertainty quantification by constructing prediction intervals or sets with finite-sample covera…
Conformalizing Machine Translation Evaluation
Several uncertainty estimation methods have been recently proposed for machine translation evaluation. While these methods can provide a useful indication of when not to trust model predictions, we show in this paper tha…
Conformal PredictionMachine TranslationPredictionTranslationCD-split and HPD-split: efficient conformal regions in high dimensions
Conformal methods create prediction bands that control average coverage assuming solely i.i.d. data. Although the literature has mostly focused on prediction intervals, more general regions can often better represent unc…
PredictionPrediction IntervalsVocal Bursts Intensity Prediction