High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approach
This paper considers the generation of prediction intervals (PIs) by neural networks for quantifying uncertainty in regression tasks. It is axiomatic that high-quality PIs should be as narrow as possible, whilst capturing a specified portion of data. We derive a loss function directly from this axiom that requires no distributional assumption. We show how its form derives from a likelihood principle, that it can be used with gradient descent, and that model uncertainty is accounted for in ensembled form. Benchmark experiments show the method outperforms current state-of-the-art uncertainty quantification methods, reducing average PI width by over 10%.
Code (1)
Tasks
FormPrediction IntervalsregressionUncertainty QuantificationSimilar Papers 제목 키워드 기반
Lifted Coefficient of Determination: Fast model-free prediction intervals and likelihood-free model comparison
We propose the $\textit{lifted linear model}$, and derive model-free prediction intervals that become tighter as the correlation between predictions and observations increases. These intervals motivate the $\textit{Lifte…
modelOutlier DetectionPredictionPrediction Intervals+1Conformalizing Machine Translation Evaluation
Several uncertainty estimation methods have been recently proposed for machine translation evaluation. While these methods can provide a useful indication of when not to trust model predictions, we show in this paper tha…
Conformal PredictionMachine TranslationPredictionTranslationDistribution-free binary classification: prediction sets, confidence intervals and calibration
We study three notions of uncertainty quantification -- calibration, confidence intervals and prediction sets -- for binary classification in the distribution-free setting, that is without making any distributional assum…
Binary ClassificationClassificationGeneral ClassificationUncertainty QuantificationPrediction Intervals: Split Normal Mixture from Quality-Driven Deep Ensembles
Prediction intervals are a machine- and human-interpretable way to represent predictive uncertainty in a regression analysis. In this paper, we present a method for generating prediction intervals along with point estima…
PredictionPrediction IntervalsBuilding Conformal Prediction Intervals with Approximate Message Passing
Conformal prediction has emerged as a powerful tool for building prediction intervals that are valid in a distribution-free way. However, its evaluation may be computationally costly, especially in the high-dimensional s…
BenchmarkingConformal PredictionPredictionPrediction Intervals+2