CRUDE: Calibrating Regression Uncertainty Distributions Empirically
Calibrated uncertainty estimates in machine learning are crucial to many fields such as autonomous vehicles, medicine, and weather and climate forecasting. While there is extensive literature on uncertainty calibration for classification, the classification findings do not always translate to regression. As a result, modern models for predicting uncertainty in regression settings typically produce uncalibrated and overconfident estimates. To address these gaps, we present a calibration method for regression settings that does not assume a particular uncertainty distribution over the error: Calibrating Regression Uncertainty Distributions Empirically (CRUDE). CRUDE makes the weaker assumption that error distributions have a constant arbitrary shape across the output space, shifted by predicted mean and scaled by predicted standard deviation. We detail a theoretical connection between CRUDE and conformal inference. Across an extensive set of regression tasks, CRUDE demonstrates consistently sharper, better calibrated, and more accurate uncertainty estimates than state-of-the-art techniques.
Code (0)
등록된 구현이 없습니다.
Tasks
Autonomous VehiclesGeneral ClassificationregressionSimilar Papers 제목 키워드 기반
Calibrating conditional risk
We introduce and study the problem of calibrating conditional risk, which involves estimating the expected loss of a prediction model conditional on input features. We analyze this problem in both classification and regr…
Forecasting the term structure of crude oil futures prices with neural networks
The paper contributes to the rare literature modeling term structure of crude oil markets. We explain term structure of crude oil prices using dynamic Nelson-Siegel model, and propose to forecast them with the generalize…
regressionEconometric Model Using Arbitrage Pricing Theory and Quantile Regression to Estimate the Risk Factors Driving Crude Oil Returns
This work adopts a novel approach to determine the risk and return of crude oil stocks by employing Arbitrage Pricing Theory (APT) and Quantile Regression (QR).The APT identifies the underlying risk factors likely to imp…
quantile regressionImpact of Economic Uncertainty, Geopolitical Risk, Pandemic, Financial & Macroeconomic Factors on Crude Oil Returns -- An Empirical Investigation
This study aims to use simultaneous quantile regression (SQR) to examine the impact of macroeconomic and financial uncertainty including global pandemic, geopolitical risk on the futures returns of crude oil (ROC). The d…
quantile regressionPost-hoc loss-calibration for Bayesian neural networks
Bayesian decision theory provides an elegant framework for acting optimally under uncertainty when tractable posterior distributions are available. Modern Bayesian models, however, typically involve intractable posterior…
Decision MakingDiversity