Quantifying Deep Learning Model Uncertainty in Conformal Prediction
Precise estimation of predictive uncertainty in deep neural networks is a critical requirement for reliable decision-making in machine learning and statistical modeling, particularly in the context of medical AI. Conformal Prediction (CP) has emerged as a promising framework for representing the model uncertainty by providing well-calibrated confidence levels for individual predictions. However, the quantification of model uncertainty in conformal prediction remains an active research area, yet to be fully addressed. In this paper, we explore state-of-the-art CP methodologies and their theoretical foundations. We propose a probabilistic approach in quantifying the model uncertainty derived from the produced prediction sets in conformal prediction and provide certified boundaries for the computed uncertainty. By doing so, we allow model uncertainty measured by CP to be compared by other uncertainty quantification methods such as Bayesian (e.g., MC-Dropout and DeepEnsemble) and Evidential approaches.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionDecision MakingDeep LearningmodelPredictionUncertainty QuantificationSimilar Papers 제목 키워드 기반
Sparse Identification of Nonlinear Dynamics with Conformal Prediction
The Sparse Identification of Nonlinear Dynamics (SINDy) is a method for discovering nonlinear dynamical system models from data. Quantifying uncertainty in SINDy models is essential for assessing their reliability, parti…
Time Series ForecastingTime Series PredictionFeature ImportanceAdaptive Bounding Box Uncertainties via Two-Step Conformal Prediction
Quantifying a model's predictive uncertainty is essential for safety-critical applications such as autonomous driving. We consider quantifying such uncertainty for multi-object detection. In particular, we leverage confo…
Autonomous DrivingConformal PredictionObjectobject-detection+2Conformal Prediction Sets Improve Human Decision Making
In response to everyday queries, humans explicitly signal uncertainty and offer alternative answers when they are unsure. Machine learning models that output calibrated prediction sets through conformal prediction mimic …
Conformal PredictionDecision MakingPredictionRobust Vision-Based Runway Detection through Conformal Prediction and Conformal mAP
We explore the use of conformal prediction to provide statistical uncertainty guarantees for runway detection in vision-based landing systems (VLS). Using fine-tuned YOLOv5 and YOLOv6 models on aerial imagery, we apply c…
Conformal Predictionobject-detectionObject DetectionPredictionEvidential Uncertainty Sets in Deep Classifiers Using Conformal Prediction
In this paper, we propose Evidential Conformal Prediction (ECP) method for image classifiers to generate the conformal prediction sets. Our method is designed based on a non-conformity score function that has its roots i…
Conformal Prediction