Bayesian Optimization with Informative Covariance
Bayesian optimization is a methodology for global optimization of unknown and expensive objectives. It combines a surrogate Bayesian regression model with an acquisition function to decide where to evaluate the objective. Typical regression models are given by Gaussian processes with stationary covariance functions. However, these functions are unable to express prior input-dependent information, including possible locations of the optimum. The ubiquity of stationary models has led to the common practice of exploiting prior information via informative mean functions. In this paper, we highlight that these models can perform poorly, especially in high dimensions. We propose novel informative covariance functions for optimization, leveraging nonstationarity to encode preferences for certain regions of the search space and adaptively promote local exploration during optimization. We demonstrate that the proposed functions can increase the sample efficiency of Bayesian optimization in high dimensions, even under weak prior information.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationGaussian Processesglobal-optimizationregressionSimilar Papers 제목 키워드 기반
Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
We present a theory-first framework that interprets inference-time adaptation in large language models (LLMs) as online Bayesian state estimation. Rather than modeling rapid adaptation as implicit optimization or meta-le…
Bayesian InferenceAccelerated Bayesian Optimization throughWeight-Prior Tuning
Bayesian optimization (BO) is a widely-used method for optimizing expensive (to evaluate) problems. At the core of most BO methods is the modeling of the objective function using a Gaussian Process (GP) whose covariance …
Bayesian OptimisationBayesian OptimizationTransfer LearningExtrinsic Bayesian Optimizations on Manifolds
We propose an extrinsic Bayesian optimization (eBO) framework for general optimization problems on manifolds. Bayesian optimization algorithms build a surrogate of the objective function by employing Gaussian processes a…
Bayesian OptimizationGaussian ProcessesvalidThe Automatic Statistician: A Relational Perspective
Gaussian Processes (GPs) provide a general and analytically tractable way of modeling complex time-varying, nonparametric functions. The Automatic Bayesian Covariance Discovery (ABCD) system constructs natural-language d…
Gaussian ProcessesTime SeriesTime Series AnalysisAdaptive posterior distributions for uncertainty analysis of covariance matrices in Bayesian inversion problems for multioutput signals
In this paper we address the problem of performing Bayesian inference for the parameters of a nonlinear multi-output model and the covariance matrix of the different output signals. We propose an adaptive importance samp…
Bayesian Inference