Adjusting Regression Models for Conditional Uncertainty Calibration
Conformal Prediction methods have finite-sample distribution-free marginal coverage guarantees. However, they generally do not offer conditional coverage guarantees, which can be important for high-stakes decisions. In this paper, we propose a novel algorithm to train a regression function to improve the conditional coverage after applying the split conformal prediction procedure. We establish an upper bound for the miscoverage gap between the conditional coverage and the nominal coverage rate and propose an end-to-end algorithm to control this upper bound. We demonstrate the efficacy of our method empirically on synthetic and real-world datasets.
Code (1)
Tasks
Conformal PredictionPredictionregressionSimilar Papers 제목 키워드 기반
Calibrating conditional risk
We introduce and study the problem of calibrating conditional risk, which involves estimating the expected loss of a prediction model conditional on input features. We analyze this problem in both classification and regr…
Can bin-wise scaling improve consistency and adaptivity of prediction uncertainty for machine learning regression ?
Binwise Variance Scaling (BVS) has recently been proposed as a post hoc recalibration method for prediction uncertainties of machine learning regression problems that is able of more efficient corrections than uniform va…
regressionBeyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantification
Among the many ways of quantifying uncertainty in a regression setting, specifying the full quantile function is attractive, as quantiles are amenable to interpretation and evaluation. A model that predicts the true cond…
regressionUncertainty QuantificationCalibration in Machine Learning Uncertainty Quantification: beyond consistency to target adaptivity
Reliable uncertainty quantification (UQ) in machine learning (ML) regression tasks is becoming the focus of many studies in materials and chemical science. It is now well understood that average calibration is insufficie…
Uncertainty QuantificationParametric and Multivariate Uncertainty Calibration for Regression and Object Detection
Reliable spatial uncertainty evaluation of object detection models is of special interest and has been subject of recent work. In this work, we review the existing definitions for uncertainty calibration of probabilistic…
object-detectionObject DetectionObject Trackingregression+1