Self-Calibrating Conformal Prediction
In machine learning, model calibration and predictive inference are essential for producing reliable predictions and quantifying uncertainty to support decision-making. Recognizing the complementary roles of point and interval predictions, we introduce Self-Calibrating Conformal Prediction, a method that combines Venn-Abers calibration and conformal prediction to deliver calibrated point predictions alongside prediction intervals with finite-sample validity conditional on these predictions. To achieve this, we extend the original Venn-Abers procedure from binary classification to regression. Our theoretical framework supports analyzing conformal prediction methods that involve calibrating model predictions and subsequently constructing conditionally valid prediction intervals on the same data, where the conditioning set or conformity scores may depend on the calibrated predictions. Real-data experiments show that our method improves interval efficiency through model calibration and offers a practical alternative to feature-conditional validity.
Code (1)
Tasks
Binary ClassificationConformal PredictionDecision MakingManagementPredictionPrediction IntervalsvalidMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Self-supervised conformal prediction for uncertainty quantification in Poisson imaging problems
Image restoration problems are often ill-posed, leading to significant uncertainty in reconstructed images. Accurately quantifying this uncertainty is essential for the reliable interpretation of reconstructed images. Ho…
Conformal PredictionDeblurringDenoisingImage Denoising+3Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration
Multimodal learning often grapples with the challenge of low-quality data, which predominantly manifests as two facets: modality imbalance and noisy corruption. While these issues are often studied in isolation, we argue…
Mitigating LLM Hallucinations via Conformal Abstention
We develop a principled procedure for determining when a large language model (LLM) should abstain from responding (e.g., by saying "I don't know") in a general domain, instead of resorting to possibly "hallucinating" a …
Conformal PredictionGenerative Question AnsweringHallucinationLanguage Modelling+4Conformal calibrators
Most existing examples of full conformal predictive systems, split-conformal predictive systems, and cross-conformal predictive systems impose severe restrictions on the adaptation of predictive distributions to the test…
Conformal PredictionAdaptive Uncertainty Quantification for Generative AI
This work is concerned with conformal prediction in contemporary applications (including generative AI) where a black-box model has been trained on data that are not accessible to the user. Mirroring split-conformal infe…
Conformal PredictionUncertainty Quantification