Validity, consonant plausibility measures, and conformal prediction
Prediction of future observations is an important and challenging problem. The two mainstream approaches for quantifying prediction uncertainty use prediction regions and predictive distributions, respectively, with the latter believed to be more informative because it can perform other prediction-related tasks. The standard notion of validity, what we refer to here as Type-1 validity, focuses on coverage probability of prediction regions, while a notion of validity relevant to the other prediction-related tasks performed by predictive distributions is lacking. Here we present a new notion, called Type-2 validity, relevant to these other prediction tasks. We establish connections between Type-2 validity and coherence properties, and show that imprecise probability considerations are required in order to achieve it. We go on to show that both types of prediction validity can be achieved by interpreting the conformal prediction output as the contour function of a consonant plausibility measure. We also offer an alternative characterization of conformal prediction, based on a new nonparametric inferential model construction, wherein the appearance of consonance is natural, and prove its validity.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionPredictionSimilar Papers 제목 키워드 기반
Conformal Prediction Regions are Imprecise Highest Density Regions
Recently, Cella and Martin proved how, under an assumption called consonance, a credal set (i.e. a closed and convex set of probabilities) can be derived from the conformal transducer associated with transductive conform…
Conformal PredictionPredictionconformalClassification: A Conformal Prediction R Package for Classification
The conformalClassification package implements Transductive Conformal Prediction (TCP) and Inductive Conformal Prediction (ICP) for classification problems. Conformal Prediction (CP) is a framework that complements the p…
BIG-bench Machine LearningClassificationConformal PredictionDiagnostic+2On some practical challenges of conformal prediction
Conformal prediction is a model-free machine learning method for constructing prediction regions at a guaranteed coverage probability level. However, a data scientist often faces three challenges in practice: (i) the det…
Inductive Conformal Prediction under Data Scarcity: Exploring the Impacts of Nonconformity Measures
Conformal prediction, which makes no distributional assumptions about the data, has emerged as a powerful and reliable approach to uncertainty quantification in practical applications. The nonconformity measure used in c…
Conformal PredictionPredictionPrediction IntervalsUncertainty QuantificationConformal e-prediction
This paper discusses a counterpart of conformal prediction for e-values, conformal e-prediction. Conformal e-prediction is conceptually simpler and had been developed in the 1990s as precursor of conformal prediction. Wh…
Conformal PredictionPrediction