Assessment of Prediction Intervals Using Uncertainty Characteristics Curves
Accurate quantification of model uncertainty has long been recognized as a fundamental requirement for trusted AI. In regression tasks, uncertainty is typically quantified using prediction intervals calibrated to an ad-hoc operating point, making evaluation and comparison across different studies relatively difficult. Our work leverages: (1) the concept of operating characteristics curves and (2) the notion of a gain over a null reference, to derive a novel operating point agnostic assessment methodology for prediction intervals. The paper defines the Uncertainty Characteristics Curve and demonstrates its utility in selected scenarios. We argue that the proposed method addresses the current need for comprehensive assessment of prediction intervals and thus represents a valuable addition to the uncertainty quantification toolbox.
Code (0)
등록된 구현이 없습니다.
Tasks
PredictionPrediction IntervalsUncertainty QuantificationSimilar Papers 제목 키워드 기반
Uncertainty Characteristics Curves: A Systematic Assessment of Prediction Intervals
Accurate quantification of model uncertainty has long been recognized as a fundamental requirement for trusted AI. In regression tasks, uncertainty is typically quantified using prediction intervals calibrated to a speci…
PredictionPrediction IntervalsregressionUncertainty QuantificationConfidence Intervals and Simultaneous Confidence Bands Based on Deep Learning
Deep learning models have significantly improved prediction accuracy in various fields, gaining recognition across numerous disciplines. Yet, an aspect of deep learning that remains insufficiently addressed is the assess…
Deep LearningUncertainty QuantificationvalidScenario-aware Uncertainty Quantification for Trajectory Prediction with Statistical Guarantees
Reliable uncertainty quantification in trajectory prediction is crucial for safety-critical autonomous driving systems, yet existing deep learning predictors lack uncertainty-aware frameworks adaptable to heterogeneous r…
Trajectory PredictionAutonomous DrivingLearning Prediction Intervals for Model Performance
Understanding model performance on unlabeled data is a fundamental challenge of developing, deploying, and maintaining AI systems. Model performance is typically evaluated using test sets or periodic manual quality asses…
modelPredictionPrediction IntervalsTransfer LearningSimultaneous Prediction Intervals for Patient-Specific Survival Curves
Accurate models of patient survival probabilities provide important information to clinicians prescribing care for life-threatening and terminal ailments. A recently developed class of models - known as individual surviv…
DescriptivePredictionPrediction IntervalsSurvival Analysis+1