LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions

2023-11-19 · E. Visser, C. E. van Daalen, J. C. Schoeman

Bayesian optimization (BO) is a popular method for optimizing expensive black-box functions. BO has several well-documented shortcomings, including computational slowdown with longer optimization runs, poor suitability for non-stationary or ill-conditioned objective functions, and poor convergence characteristics. Several algorithms have been proposed that incorporate local strategies, such as trust regions, into BO to mitigate these limitations; however, none address all of them satisfactorily. To address these shortcomings, we propose the LABCAT algorithm, which extends trust-region-based BO by adding a rotation aligning the trust region with the weighted principal components and an adaptive rescaling strategy based on the length-scales of a local Gaussian process surrogate model with automatic relevance determination. Through extensive numerical experiments using a set of synthetic test functions and the well-known COCO benchmarking software, we show that the LABCAT algorithm outperforms several state-of-the-art BO and other black-box optimization algorithms.

📄 PDF Abstract BibTeX arXiv:2311.11328

Code (2)

aemiliusretiarius/labcat 공식 구현

esl-sun/labcat 공식 구현

Tasks

Bayesian OptimizationBenchmarking

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Adaptive probabilistic principal component analysis

2019-05-27 · Adam Farooq, Yordan P. Raykov, Luc Evers, Max A. Little

Using the linear Gaussian latent variable model as a starting point we relax some of the constraints it imposes by deriving a nonparametric latent feature Gaussian variable model. This model introduces additional discret…

Local Kernels that Approximate Bayesian Regularization and Proximal Operators

2018-03-09 · Frank Ong, Peyman Milanfar, Pascal Getreuer

In this work, we broadly connect kernel-based filtering (e.g. approaches such as the bilateral filters and nonlocal means, but also many more) with general variational formulations of Bayesian regularized least squares, …

global-optimization valid

LinEasyBO: Scalable Bayesian Optimization Approach for Analog Circuit Synthesis via One-Dimensional Subspaces

2021-09-01 · Shuhan Zhang, Fan Yang, Changhao Yan, Dian Zhou 외

A large body of literature has proved that the Bayesian optimization framework is especially efficient and effective in analog circuit synthesis. However, most of the previous research works only focus on designing infor…

Bayesian Optimization

Lessons from AlphaZero for Optimal, Model Predictive, and Adaptive Control

2021-08-20 · Dimitri Bertsekas

In this paper we aim to provide analysis and insights (often based on visualization), which explain the beneficial effects of on-line decision making on top of off-line training. In particular, through a unifying abstrac…

Bayesian Optimization Decision Making Model Predictive Control

Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes

2026-01-27 · Yan Zhang, Xuefeng Liu, Sipeng Chen, Sascha Ranftl 외 arxiv

Standard Bayesian Optimization (BO) assumes uniform smoothness across the search space an assumption violated in multi-regime problems such as molecular conformation search through distinct energy basins or drug discover…

Gaussian Processes Drug Discovery