Frustratingly Easy Uncertainty Estimation for Distribution Shift
Distribution shift is an important concern in deep image classification, produced either by corruption of the source images, or a complete change, with the solution involving domain adaptation. While the primary goal is to improve accuracy under distribution shift, an important secondary goal is uncertainty estimation: evaluating the probability that the prediction of a model is correct. While improving accuracy is hard, uncertainty estimation turns out to be frustratingly easy. Prior works have appended uncertainty estimation into the model and training paradigm in various ways. Instead, we show that we can estimate uncertainty by simply exposing the original model to corrupted images, and performing simple statistical calibration on the image outputs. Our frustratingly easy methods demonstrate superior performance on a wide range of distribution shifts as well as on unsupervised domain adaptation tasks, measured through extensive experimentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain Adaptationimage-classificationImage ClassificationUnsupervised Domain AdaptationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Return of Frustratingly Easy Domain Adaptation
Unlike human learning, machine learning often fails to handle changes between training (source) and test (target) input distributions. Such domain shifts, common in practical scenarios, severely damage the performance of…
BIG-bench Machine LearningDomain AdaptationUnsupervised Domain AdaptationDeep CORAL: Correlation Alignment for Deep Domain Adaptation
Deep neural networks are able to learn powerful representations from large quantities of labeled input data, however they cannot always generalize well across changes in input distributions. Domain adaptation algorithms …
Domain AdaptationDomain GeneralizationImage ClassificationUnsupervised Domain AdaptationFrustratingly Easy Cross-Lingual Transfer for Transition-Based Dependency Parsing
Temporal Distribution Shift in Real-World Pharmaceutical Data: Implications for Uncertainty Quantification in QSAR Models
The estimation of uncertainties associated with predictions from quantitative structure-activity relationship (QSAR) models can accelerate the drug discovery process by identifying promising experiments and allowing an e…
Drug DiscoveryUncertainty QuantificationHow Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?
Many important computer vision applications are naturally formulated as regression problems. Within medical imaging, accurate regression models have the potential to automate various tasks, helping to lower costs and imp…
Out-of-Distribution Detectionregression