Epistemic Reject Option Prediction
In high-stakes applications, predictive models must not only produce accurate predictions but also quantify and communicate their uncertainty. Reject-option prediction addresses this by allowing the model to abstain when prediction uncertainty is high. Traditional reject-option approaches focus solely on aleatoric uncertainty, an assumption valid only when large training data makes the epistemic uncertainty negligible. However, in many practical scenarios, limited data makes this assumption unrealistic. This paper introduces the epistemic reject-option predictor, which abstains in regions of high epistemic uncertainty caused by insufficient data. Building on Bayesian learning, we redefine the optimal predictor as the one that minimizes expected regret -- the performance gap between the learned model and the Bayes-optimal predictor with full knowledge of the data distribution. The model abstains when the regret for a given input exceeds a specified rejection cost. To our knowledge, this is the first principled framework that enables learning predictors capable of identifying inputs for which the available training data is insufficient to support well-informed predictions.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Out-of-distribution Reject Option Method for Dataset Shift Problem in Early Disease Onset Prediction
Machine learning is increasingly used to predict lifestyle-related disease onset using health and medical data. However, its predictive accuracy for use is often hindered by dataset shift, which refers to discrepancies i…
Disease PredictionPredictionEvaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning
Current evaluation of epistemic uncertainty relies on tasks such as out-ofdistribution detection and active learning. However, the Bayes-optimal decision strategies for these tasks do not coincide with the scores commonl…
Active LearningBayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection
Despite numerous studies of deep autoencoders (AEs) for unsupervised anomaly detection, AEs still lack a way to express uncertainty in their predictions, crucial for ensuring safe and trustworthy machine learning systems…
Anomaly DetectionUncertainty QuantificationUnsupervised Anomaly DetectionModel Agnostic Local Explanations of Reject
The application of machine learning based decision making systems in safety critical areas requires reliable high certainty predictions. Reject options are a common way of ensuring a sufficiently high certainty of predic…
counterfactualDecision Makingmodel"Even if ..." -- Diverse Semifactual Explanations of Reject
Machine learning based decision making systems applied in safety critical areas require reliable high certainty predictions. For this purpose, the system can be extended by an reject option which allows the system to rej…
BIG-bench Machine LearningConformal PredictionDecision MakingExplainable Artificial Intelligence (XAI)