Non-exchangeable Conformal Prediction with Optimal Transport: Tackling Distribution Shifts with Unlabeled Data

2025-07-14 · Alvaro H. C. Correia, Christos Louizos arxiv

Conformal prediction is a distribution-free uncertainty quantification method that has gained popularity in the machine learning community due to its finite-sample guarantees and ease of use. Its most common variant, dubbed split conformal prediction, is also computationally efficient as it boils down to collecting statistics of the model predictions on some calibration data not yet seen by the model. Nonetheless, these guarantees only hold if the calibration and test data are exchangeable, a condition that is difficult to verify and often violated in practice due to so-called distribution shifts. The literature is rife with methods to mitigate the loss in coverage in this non-exchangeable setting, but these methods require some prior information on the type of distribution shift to be expected at test time. In this work, we study this problem via a new perspective, through the lens of optimal transport, and show that it is possible to estimate the loss in coverage and mitigate arbitrary distribution shifts, offering a principled and broadly applicable solution.

📄 PDF Abstract BibTeX arXiv:2507.10425

Code (0)

등록된 구현이 없습니다.

Efficient Conformal Prediction under Data Heterogeneity

2023-12-25 · Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov 외

Conformal Prediction (CP) stands out as a robust framework for uncertainty quantification, which is crucial for ensuring the reliability of predictions. However, common CP methods heavily rely on data exchangeability, a …

Conformal Prediction Federated Learning Prediction Uncertainty Quantification+1

Non-Exchangeable Conformal Language Generation with Nearest Neighbors

2024-02-01 · Dennis Ulmer, Chrysoula Zerva, André F. T. Martins

Quantifying uncertainty in automatically generated text is important for letting humans check potential hallucinations and making systems more reliable. Conformal prediction is an attractive framework to provide predicti…

Conformal Prediction Language Modeling Language Modelling Machine Translation+2

Kernel-based Optimally Weighted Conformal Prediction Intervals

2024-05-27 · JongHyeok Lee, Chen Xu, Yao Xie

In this work, we present a novel conformal prediction method for time-series, which we call Kernel-based Optimally Weighted Conformal Prediction Intervals (KOWCPI). Specifically, KOWCPI adapts the classic Reweighted Nada…

Conformal Prediction Prediction Prediction Intervals quantile regression+2

Extending Conformal Prediction to Hidden Markov Models with Exact Validity via de Finetti's Theorem for Markov Chains

2022-10-05 · Buddhika Nettasinghe, Samrat Chatterjee, Ramakrishna Tipireddy, Mahantesh Halappanavar

Conformal prediction is a widely used method to quantify the uncertainty of a classifier under the assumption of exchangeability (e.g., IID data). We generalize conformal prediction to the Hidden Markov Model (HMM) frame…

Conformal Prediction Prediction valid

Beyond Conformal Predictors: Adaptive Conformal Inference with Confidence Predictors

2024-09-23 · Johan Hallberg Szabadváry

Conformal prediction (CP) is a robust framework for distribution-free uncertainty quantification, but it requires exchangeable data to ensure valid prediction sets at a user-specified significance level. When this assump…

Computational Efficiency Conformal Prediction Prediction Uncertainty Quantification+1