Improving the statistical efficiency of cross-conformal prediction
Vovk (2015) introduced cross-conformal prediction, a modification of split conformal designed to improve the width of prediction sets. The method, when trained with a miscoverage rate equal to $\alpha$ and $n \gg K$, ensures a marginal coverage of at least $1 - 2\alpha - 2(1-\alpha)(K-1)/(n+K)$, where $n$ is the number of observations and $K$ denotes the number of folds. A simple modification of the method achieves coverage of at least $1-2\alpha$. In this work, we propose new variants of both methods that yield smaller prediction sets without compromising the latter theoretical guarantees. The proposed methods are based on recent results deriving more statistically efficient combination of p-values that leverage exchangeability and randomization. Simulations confirm the theoretical findings and bring out some important tradeoffs.
Code (1)
Tasks
Conformal PredictionPredictionSimilar Papers 제목 키워드 기반
Uncertainty Quantification in Anomaly Detection with Cross-Conformal $p$-Values
Given the growing significance of reliable, trustworthy, and explainable machine learning, the requirement of uncertainty quantification for anomaly detection systems has become increasingly important. In this context, e…
Anomaly DetectionConformal PredictionPredictionUncertainty QuantificationStatistical Guarantees in Synthetic Data through Conformal Adversarial Generation
The generation of high-quality synthetic data presents significant challenges in machine learning research, particularly regarding statistical fidelity and uncertainty quantification. Existing generative models produce c…
Conformal PredictionMathematical ProofsPredictionUncertainty QuantificationSet-Preserving Calibration from Conformal P-Values to E-Values
Standard conformal prediction (CP) procedures are typically formulated in terms of p-values, but reliance on p-values alone limits flexibility, for example, when combining dependent evidence across models or data splits.…
Conformal Correction for Efficiency May be at Odds with Entropy
Conformal prediction (CP) provides a comprehensive framework to produce statistically rigorous uncertainty sets for black-box machine learning models. To further improve the efficiency of CP, conformal correction is prop…
Online Conformal Prediction: Enforcing monotonicity via Online Optimization
Conformal prediction provides a principled framework for uncertainty quantification with finite-sample coverage guarantees. While recent work has extended conformal prediction to online and sequential settings, existing …
Weather Forecasting