Is the Volume of a Credal Set a Good Measure for Epistemic Uncertainty?
Adequate uncertainty representation and quantification have become imperative in various scientific disciplines, especially in machine learning and artificial intelligence. As an alternative to representing uncertainty via one single probability measure, we consider credal sets (convex sets of probability measures). The geometric representation of credal sets as $d$-dimensional polytopes implies a geometric intuition about (epistemic) uncertainty. In this paper, we show that the volume of the geometric representation of a credal set is a meaningful measure of epistemic uncertainty in the case of binary classification, but less so for multi-class classification. Our theoretical findings highlight the crucial role of specifying and employing uncertainty measures in machine learning in an appropriate way, and for being aware of possible pitfalls.
Code (0)
등록된 구현이 없습니다.
Tasks
Binary ClassificationMulti-class ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Credal Two-Sample Tests of Epistemic Uncertainty
We introduce credal two-sample testing, a new hypothesis testing framework for comparing credal sets -- convex sets of probability measures where each element captures aleatoric uncertainty and the set itself represents …
Two-sample testingQuantification of Credal Uncertainty: A Distance-Based Approach
Credal sets, i.e., closed convex sets of probability measures, provide a natural framework to represent aleatoric and epistemic uncertainty in machine learning. Yet how to quantify these two types of uncertainty for a gi…
Credal Concept Bottleneck Models for Epistemic-Aleatoric Uncertainty Decomposition
Concept Bottleneck Models (CBMs) predict through human-interpretable concepts, but they typically output point concept probabilities that conflate epistemic uncertainty (reducible model underspecification) with aleatoric…
Ensemble-based Uncertainty Quantification: Bayesian versus Credal Inference
The idea to distinguish and quantify two important types of uncertainty, often referred to as aleatoric and epistemic, has received increasing attention in machine learning research in the last couple of years. In this p…
Ensemble LearningUncertainty QuantificationQuantifying Epistemic Predictive Uncertainty in Conformal Prediction
We study the problem of quantifying epistemic predictive uncertainty (EPU) -- that is, uncertainty faced at prediction time due to the existence of multiple plausible predictive models -- within the framework of conforma…
Active Learning