Self-Correcting Bayesian Optimization through Bayesian Active Learning
Gaussian processes are the model of choice in Bayesian optimization and active learning. Yet, they are highly dependent on cleverly chosen hyperparameters to reach their full potential, and little effort is devoted to finding good hyperparameters in the literature. We demonstrate the impact of selecting good hyperparameters for GPs and present two acquisition functions that explicitly prioritize hyperparameter learning. Statistical distance-based Active Learning (SAL) considers the average disagreement between samples from the posterior, as measured by a statistical distance. SAL outperforms the state-of-the-art in Bayesian active learning on several test functions. We then introduce Self-Correcting Bayesian Optimization (SCoreBO), which extends SAL to perform Bayesian optimization and active learning simultaneously. SCoreBO learns the model hyperparameters at improved rates compared to vanilla BO, while outperforming the latest Bayesian optimization methods on traditional benchmarks. Moreover, we demonstrate the importance of self-correction on atypical Bayesian optimization tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Active LearningBayesian OptimizationGaussian ProcessesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Bayesian Calibration of MEMS Accelerometers
This study aims to investigate the utilization of Bayesian techniques for the calibration of micro-electro-mechanical systems (MEMS) accelerometers. These devices have garnered substantial interest in various practical a…
Probabilistic ProgrammingYou Only Accept Samples Once: Fast, Self-Correcting Stochastic Variational Inference
We introduce YOASOVI, an algorithm for performing fast, self-correcting stochastic optimization for Variational Inference (VI) on large Bayesian heirarchical models. To accomplish this, we take advantage of available inf…
Stochastic OptimizationVariational InferencePost-hoc loss-calibration for Bayesian neural networks
Bayesian decision theory provides an elegant framework for acting optimally under uncertainty when tractable posterior distributions are available. Modern Bayesian models, however, typically involve intractable posterior…
Decision MakingDiversityCorrecting Predictions for Approximate Bayesian Inference
Bayesian models quantify uncertainty and facilitate optimal decision-making in downstream applications. For most models, however, practitioners are forced to use approximate inference techniques that lead to sub-optimal …
Bayesian InferenceDecision MakingBayesian Self-Supervised Contrastive Learning
Recent years have witnessed many successful applications of contrastive learning in diverse domains, yet its self-supervised version still remains many exciting challenges. As the negative samples are drawn from unlabele…
Contrastive Learning