Derivation of Closed Form of Expected Improvement for Gaussian Process Trained on Log-Transformed Objective

2024-11-27 · Shuhei Watanabe

Expected Improvement (EI) is arguably the most widely used acquisition function in Bayesian optimization. However, it is often challenging to enhance the performance with EI due to its sensitivity to numerical precision. Previously, Hutter et al. (2009) tackled this problem by using Gaussian process trained on the log-transformed objective function and it was reported that this trick improves the predictive accuracy of GP, leading to substantially better performance. Although Hutter et al. (2009) offered the closed form of their EI, its intermediate derivation has not been provided so far. In this paper, we give a friendly derivation of their proposition.

📄 PDF Abstract BibTeX arXiv:2411.18095

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian OptimizationFormSensitivity

Methods 이 논문이 사용한 방법론

Gaussian Process Gaussian Processes are non-parametric models for approximating functions. They rely upon a measure of similarity between points (the kernel function) to predict the value for…

Efficient batch-sequential Bayesian optimization with moments of truncated Gaussian vectors

2016-09-09 · Sébastien Marmin, Clément Chevalier, David Ginsbourger

We deal with the efficient parallelization of Bayesian global optimization algorithms, and more specifically of those based on the expected improvement criterion and its variants. A closed form formula relying on multiva…

Bayesian Optimization Form global-optimization

Differentiating the multipoint Expected Improvement for optimal batch design

2015-03-18 · Sébastien Marmin, Clément Chevalier, David Ginsbourger

This work deals with parallel optimization of expensive objective functions which are modeled as sample realizations of Gaussian processes. The study is formalized as a Bayesian optimization problem, or continuous multi-…

Bayesian Optimization Gaussian Processes

Bayesian Optimization of Composite Functions

2019-06-04 · Raul Astudillo, Peter I. Frazier

We consider optimization of composite objective functions, i.e., of the form $f(x)=g(h(x))$, where $h$ is a black-box derivative-free expensive-to-evaluate function with vector-valued outputs, and $g$ is a cheap-to-evalu…

Bayesian Optimization

KL Divergence Between Gaussians: A Step-by-Step Derivation for the Variational Autoencoder Objective

2026-04-13 · Andrés Muñoz, Rodrigo Ramele arxiv

Kullback-Leibler (KL) divergence is a fundamental concept in information theory that quantifies the discrepancy between two probability distributions. In the context of Variational Autoencoders (VAEs), it serves as a cen…

Upgrading from Gaussian Processes to Student's-T Processes

2018-01-18 · Brendan D. Tracey, David H. Wolpert

Gaussian process priors are commonly used in aerospace design for performing Bayesian optimization. Nonetheless, Gaussian processes suffer two significant drawbacks: outliers are a priori assumed unlikely, and the poster…