paper-with-me

홈 › Papers

Self-Calibrating Conformal Prediction

2024-02-11 · Lars van der Laan, Ahmed M. Alaa

In machine learning, model calibration and predictive inference are essential for producing reliable predictions and quantifying uncertainty to support decision-making. Recognizing the complementary roles of point and interval predictions, we introduce Self-Calibrating Conformal Prediction, a method that combines Venn-Abers calibration and conformal prediction to deliver calibrated point predictions alongside prediction intervals with finite-sample validity conditional on these predictions. To achieve this, we extend the original Venn-Abers procedure from binary classification to regression. Our theoretical framework supports analyzing conformal prediction methods that involve calibrating model predictions and subsequently constructing conditionally valid prediction intervals on the same data, where the conditioning set or conformity scores may depend on the calibrated predictions. Real-data experiments show that our method improves interval efficiency through model calibration and offers a practical alternative to feature-conditional validity.

📄 PDF Abstract BibTeX arXiv:2402.07307

Code (1)

larsvanderlaan/selfcalibratingconformal 공식 구현

Tasks

Binary ClassificationConformal PredictionDecision MakingManagementPredictionPrediction Intervalsvalid

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Similar Papers 제목 키워드 기반

Self-supervised conformal prediction for uncertainty quantification in Poisson imaging problems

2025-02-26 · Bernardin Tamo Amougou, Marcelo Pereyra, Barbara Pascal

Image restoration problems are often ill-posed, leading to significant uncertainty in reconstructed images. Accurately quantifying this uncertainty is essential for the reliable interpretation of reconstructed images. Ho…

Conformal PredictionDeblurringDenoisingImage Denoising+3

Multimodal Learning on Low-Quality Data with Conformal Predictive Self-Calibration

2026-05-05 · Xun Jiang, Yufan Gu, Disen Hu, Yuqing Hou 외 arxiv

Multimodal learning often grapples with the challenge of low-quality data, which predominantly manifests as two facets: modality imbalance and noisy corruption. While these issues are often studied in isolation, we argue…

Mitigating LLM Hallucinations via Conformal Abstention

2024-04-04 · Yasin Abbasi Yadkori, Ilja Kuzborskij, David Stutz, András György 외

We develop a principled procedure for determining when a large language model (LLM) should abstain from responding (e.g., by saying "I don't know") in a general domain, instead of resorting to possibly "hallucinating" a …

Conformal PredictionGenerative Question AnsweringHallucinationLanguage Modelling+4

Conformal calibrators

2019-02-18 · Vladimir Vovk, Ivan Petej, Paolo Toccaceli, Alex Gammerman

Most existing examples of full conformal predictive systems, split-conformal predictive systems, and cross-conformal predictive systems impose severe restrictions on the adaptation of predictive distributions to the test…

Conformal Prediction

Adaptive Uncertainty Quantification for Generative AI

2024-08-16 · Jungeum Kim, Sean O'Hagan, Veronika Rockova

This work is concerned with conformal prediction in contemporary applications (including generative AI) where a black-box model has been trained on data that are not accessible to the user. Mirroring split-conformal infe…

Conformal PredictionUncertainty Quantification