Coverage-Guaranteed Prediction Sets for Out-of-Distribution Data
Out-of-distribution (OOD) generalization has attracted increasing research attention in recent years, due to its promising experimental results in real-world applications. In this paper,we study the confidence set prediction problem in the OOD generalization setting. Split conformal prediction (SCP) is an efficient framework for handling the confidence set prediction problem. However, the validity of SCP requires the examples to be exchangeable, which is violated in the OOD setting. Empirically, we show that trivially applying SCP results in a failure to maintain the marginal coverage when the unseen target domain is different from the source domain. To address this issue, we develop a method for forming confident prediction sets in the OOD setting and theoretically prove the validity of our method. Finally, we conduct experiments on simulated data to empirically verify the correctness of our theory and the validity of our proposed method.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionPredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Robust Conformal Prediction under Distribution Shift via Physics-Informed Structural Causal Model
Uncertainty is critical to reliable decision-making with machine learning. Conformal prediction (CP) handles uncertainty by predicting a set on a test input, hoping the set to cover the true label with at least $(1-\alph…
Conformal PredictionDecision MakingProbabilistic Conformal Coverage Guarantees in Small-Data Settings
Conformal prediction provides distribution-free prediction sets with guaranteed marginal coverage. However, in split conformal prediction this guarantee is training-conditional only in expectation: across many calibratio…
Robust Conformal Prediction Using Privileged Information
We develop a method to generate prediction sets with a guaranteed coverage rate that is robust to corruptions in the training data, such as missing or noisy variables. Our approach builds on conformal prediction, a power…
Conformal PredictionPredictionvalidWasserstein-regularized Conformal Prediction under General Distribution Shift
Conformal prediction yields a prediction set with guaranteed $1-\alpha$ coverage of the true target under the i.i.d. assumption, which may not hold and lead to a gap between $1-\alpha$ and the actual coverage. Prior stud…
Conformal PredictionPredictionRepresentation LearningMulti-Distribution Robust Conformal Prediction
In many fairness and distribution robustness problems, one has access to labeled data from multiple source distributions yet the test data may come from an arbitrary member or a mixture of them. We study the problem of c…