Assessing Reliability and Challenges of Uncertainty Estimations for Medical Image Segmentation
Despite the recent improvements in overall accuracy, deep learning systems still exhibit low levels of robustness. Detecting possible failures is critical for a successful clinical integration of these systems, where each data point corresponds to an individual patient. Uncertainty measures are a promising direction to improve failure detection since they provide a measure of a system's confidence. Although many uncertainty estimation methods have been proposed for deep learning, little is known on their benefits and current challenges for medical image segmentation. Therefore, we report results of evaluating common voxel-wise uncertainty measures with respect to their reliability, and limitations on two medical image segmentation datasets. Results show that current uncertainty methods perform similarly and although they are well-calibrated at the dataset level, they tend to be miscalibrated at subject-level. Therefore, the reliability of uncertainty estimates is compromised, highlighting the importance of developing subject-wise uncertainty estimations. Additionally, among the benchmarked methods, we found auxiliary networks to be a valid alternative to common uncertainty methods since they can be applied to any previously trained segmentation model.
Code (1)
Tasks
Image SegmentationMedical Image SegmentationSegmentationSemantic SegmentationvalidSimilar Papers 제목 키워드 기반
Uncertainty estimations methods for a deep learning model to aid in clinical decision-making -- a clinician's perspective
Prediction uncertainty estimation has clinical significance as it can potentially quantify prediction reliability. Clinicians may trust 'blackbox' models more if robust reliability information is available, which may lea…
Decision MakingSensitivitySpecificityVariational InferenceRDeepSense: Reliable Deep Mobile Computing Models with Uncertainty Estimations
Recent advances in deep learning have led various applications to unprecedented achievements, which could potentially bring higher intelligence to a broad spectrum of mobile and ubiquitous applications. Although existing…
scoring ruleMeasuring and Modeling Uncertainty Degree for Monocular Depth Estimation
Effectively measuring and modeling the reliability of a trained model is essential to the real-world deployment of monocular depth estimation (MDE) models. However, the intrinsic ill-posedness and ordinal-sensitive natur…
Depth EstimationMonocular Depth EstimationUAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
Despite demonstrating impressive capabilities, Large Language Models (LLMs) still often struggle to accurately express the factual knowledge they possess, especially in cases where the LLMs' knowledge boundaries are ambi…
Question AnsweringA Deep Bayesian Convolutional Spiking Neural Network-based CAD system with Uncertainty Quantification for Medical Images Classification
The Computer_Aided Diagnosis (CAD) systems facilitate accurate diagnosis of diseases. The development of CADs by leveraging third generation neural network, namely, Spiking Neural Network (SNN), is essential to utilize o…
Deep Learningimage-classificationImage ClassificationMedical Image Classification+1