Credal Ensemble Distillation for Uncertainty Quantification
Deep ensembles (DE) have emerged as a powerful approach for quantifying predictive uncertainty and distinguishing its aleatoric and epistemic components, thereby enhancing model robustness and reliability. However, their high computational and memory costs during inference pose significant challenges for wide practical deployment. To overcome this issue, we propose credal ensemble distillation (CED), a novel framework that compresses a DE into a single model, CREDIT, for classification tasks. Instead of a single softmax probability distribution, CREDIT predicts class-wise probability intervals that define a credal set, a convex set of probability distributions, for uncertainty quantification. Empirical results on out-of-distribution detection benchmarks demonstrate that CED achieves superior or comparable uncertainty estimation compared to several existing baselines, while substantially reducing inference overhead compared to DE.
Code (0)
등록된 구현이 없습니다.
Tasks
Out-of-Distribution DetectionSimilar Papers 제목 키워드 기반
Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference
The idea to distinguish and quantify two important types of uncertainty, often referred to as aleatoric and epistemic, has received increasing attention in machine learning research in the last couple of years. In this p…
Ensemble LearningUncertainty QuantificationCredal Graph Neural Networks
Uncertainty quantification is essential for deploying reliable Graph Neural Networks (GNNs), where existing approaches primarily rely on Bayesian inference or ensembles. In this paper, we introduce the first credal graph…
Node ClassificationBayesian InferenceDistilling Calibration via Conformalized Credal Inference
Deploying artificial intelligence (AI) models on edge devices involves a delicate balance between meeting stringent complexity constraints, such as limited memory and energy resources, and ensuring reliable performance i…
Bayesian InferenceUncertainty QuantificationCreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks
Effective uncertainty estimation is becoming increasingly attractive for enhancing the reliability of neural networks. This work presents a novel approach, termed Credal-Set Interval Neural Networks (CreINNs), for classi…
Binary ClassificationOut-of-Distribution DetectionUncertainty QuantificationIs the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?
Adequate uncertainty representation and quantification have become imperative in various scientific disciplines, especially in machine learning and artificial intelligence. As an alternative to representing uncertainty v…
Binary ClassificationMulti-class Classification