Conformal Risk Control
We extend conformal prediction to control the expected value of any monotone loss function. The algorithm generalizes split conformal prediction together with its coverage guarantee. Like conformal prediction, the conformal risk control procedure is tight up to an $\mathcal{O}(1/n)$ factor. We also introduce extensions of the idea to distribution shift, quantile risk control, multiple and adversarial risk control, and expectations of U-statistics. Worked examples from computer vision and natural language processing demonstrate the usage of our algorithm to bound the false negative rate, graph distance, and token-level F1-score.
Code (2)
Tasks
Conformal PredictionPredictionSimilar Papers 제목 키워드 기반
Conformal Risk Control for Ordinal Classification
As a natural extension to the standard conformal prediction method, several conformal risk control methods have been recently developed and applied to various learning problems. In this work, we seek to control the confo…
ClassificationConformal PredictionDiabetic Retinopathy DetectionOrdinal ClassificationConditional Conformal Risk Adaptation
Uncertainty quantification is becoming increasingly important in image segmentation, especially for high-stakes applications like medical imaging. While conformal risk control generalizes conformal prediction beyond stan…
Conformal PredictionImage SegmentationPersonalized SegmentationSegmentation+2Conformal Risk Training: End-to-End Optimization of Conformal Risk Control
While deep learning models often achieve high predictive accuracy, their predictions typically do not come with any provable guarantees on risk or reliability, which are critical for deployment in high-stakes application…
Conformal Risk Control for Non-Monotonic Losses
Conformal risk control is an extension of conformal prediction for controlling risk functions beyond miscoverage. The original algorithm controls the expected value of a loss that is monotonic in a one-dimensional parame…
Image ClassificationTraining-Aware Risk Control for Intensity Modulated Radiation Therapies Quality Assurance with Conformal Prediction
Measurement quality assurance (QA) practices play a key role in the safe use of Intensity Modulated Radiation Therapies (IMRT) for cancer treatment. These practices have reduced measurement-based IMRT QA failure below 1%…
Conformal PredictionDecision MakingSpecificity