Simulator Calibration under Covariate Shift with Kernels
We propose a novel calibration method for computer simulators, dealing with the problem of covariate shift. Covariate shift is the situation where input distributions for training and test are different, and ubiquitous in applications of simulations. Our approach is based on Bayesian inference with kernel mean embedding of distributions, and on the use of an importance-weighted reproducing kernel for covariate shift adaptation. We provide a theoretical analysis for the proposed method, including a novel theoretical result for conditional mean embedding, as well as empirical investigations suggesting its effectiveness in practice. The experiments include calibration of a widely used simulator for industrial manufacturing processes, where we also demonstrate how the proposed method may be useful for sensitivity analysis of model parameters.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceSimilar Papers 제목 키워드 기반
Expectation Consistency Loss: Rethink Confidence Calibration under Covariate Shift
Confidence calibration for classification models is vital in safety-critical decision-making scenarios and has received extensive attention. General confidence calibration methods assume training and test data are indepe…
Unsupervised Domain AdaptationBridging Multicalibration and Out-of-distribution Generalization Beyond Covariate Shift
We establish a new model-agnostic optimization framework for out-of-distribution generalization via multicalibration, a criterion that ensures a predictor is calibrated across a family of overlapping groups. Multicalibra…
Out-of-Distribution GeneralizationExploring Covariate and Concept Shift for Detection and Confidence Calibration of Out-of-Distribution Data
Moving beyond testing on in-distribution data, works on Out-of-Distribution (OOD) detection have recently increased in popularity. A recent attempt to categorize OOD data introduces the concept of near and far OOD detect…
Out of Distribution (OOD) DetectionSensitivityExploring Covariate and Concept Shift for Detection and Calibration of Out-of-Distribution Data
Moving beyond testing on in-distribution data works on Out-of-Distribution (OOD) detection have recently increased in popularity. A recent attempt to categorize OOD data introduces the concept of near and far OOD detecti…
Out of Distribution (OOD) DetectionSensitivityUnsupervised Calibration under Covariate Shift
A probabilistic model is said to be calibrated if its predicted probabilities match the corresponding empirical frequencies. Calibration is important for uncertainty quantification and decision making in safety-critical …
Decision MakingDomain AdaptationUncertainty Quantification