One Sample is Enough to Make Conformal Prediction Robust
Given any model, conformal prediction (CP) returns prediction sets guaranteed to include the true label with high adjustable probability. Robust CP (RCP) extends this to inputs with worst-case noise. A well-established approach is to use randomized smoothing for RCP since it is applicable to any black-box model and provides smaller sets compared to deterministic methods. However, current smoothing-based RCP requires many model forward passes per each input which is computationally expensive. We show that conformal prediction attains some robustness even with a forward pass on a single randomly perturbed input. Using any binary certificate we propose a single sample robust CP (RCP1). Our approach returns robust sets with smaller average set size compared to SOTA methods which use many (e.g. around 100) passes per input. Our key insight is to certify the conformal prediction procedure itself rather than individual scores. Our approach is agnostic to the setup (classification and regression). We further extend our approach to smoothing-based robust conformal risk control.
Code (0)
등록된 구현이 없습니다.
Tasks
Conformal PredictionPredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Selecting informative conformal prediction sets with false coverage rate control
In supervised learning, including regression and classification, conformal methods provide prediction sets for the outcome/label with finite sample coverage for any machine learning predictor. We consider here the case w…
Conformal PredictionPredictionPrediction IntervalsregressionConformal prediction for text infilling and part-of-speech prediction
Modern machine learning algorithms are capable of providing remarkably accurate point-predictions; however, questions remain about their statistical reliability. Unlike conventional machine learning methods, conformal pr…
Conformal PredictionPOSPOS TaggingPrediction+2Conformal Prediction for Causal Effects of Continuous Treatments
Uncertainty quantification of causal effects is crucial for safety-critical applications such as personalized medicine. A powerful approach for this is conformal prediction, which has several practical benefits due to mo…
Conformal PredictionPredictionPrediction IntervalsUncertainty Quantification+1Conformal Predictions for Probabilistically Robust Scalable Machine Learning Classification
Conformal predictions make it possible to define reliable and robust learning algorithms. But they are essentially a method for evaluating whether an algorithm is good enough to be used in practice. To define a reliable …
Learning TheoryDistribution-Free Conformal Prediction for Steel Fatigue Strength: Marginal Validity Is Not Enough
Predicting fatigue failure in steel components experimentally is costly, requiring testing across multiple compositions and processing conditions, spurring research on data-driven prediction models. Studies using the NIM…