Max-value Entropy Search for Efficient Bayesian Optimization
Entropy Search (ES) and Predictive Entropy Search (PES) are popular and empirically successful Bayesian Optimization techniques. Both rely on a compelling information-theoretic motivation, and maximize the information gained about the $\arg\max$ of the unknown function; yet, both are plagued by the expensive computation for estimating entropies. We propose a new criterion, Max-value Entropy Search (MES), that instead uses the information about the maximum function value. We show relations of MES to other Bayesian optimization methods, and establish a regret bound. We observe that MES maintains or improves the good empirical performance of ES/PES, while tremendously lightening the computational burden. In particular, MES is much more robust to the number of samples used for computing the entropy, and hence more efficient for higher dimensional problems.
Code (4)
Tasks
Bayesian OptimisationBayesian OptimizationSimilar Papers 제목 키워드 기반
Joint Entropy Search for Maximally-Informed Bayesian Optimization
Information-theoretic Bayesian optimization techniques have become popular for optimizing expensive-to-evaluate black-box functions due to their non-myopic qualities. Entropy Search and Predictive Entropy Search both con…
Bayesian OptimizationDecision MakingImproved Max-value Entropy Search for Multi-objective Bayesian Optimization with Constraints
We present MESMOC+, an improved version of Max-value Entropy search for Multi-Objective Bayesian optimization with Constraints (MESMOC). MESMOC+ can be used to solve constrained multi-objective problems when the objectiv…
Bayesian OptimizationVariational Entropy Search for Adjusting Expected Improvement
Bayesian optimization is a widely used technique for optimizing black-box functions, with Expected Improvement (EI) being the most commonly utilized acquisition function in this domain. While EI is often viewed as distin…
Bayesian OptimizationVariational InferenceMulti-objective Bayesian Optimization using Pareto-frontier Entropy
This paper studies an entropy-based multi-objective Bayesian optimization (MBO). The entropy search is successful approach to Bayesian optimization. However, for MBO, existing entropy-based methods ignore trade-off among…
Bayesian OptimizationConstrained Bayesian Optimization with Max-Value Entropy Search
Bayesian optimization (BO) is a model-based approach to sequentially optimize expensive black-box functions, such as the validation error of a deep neural network with respect to its hyperparameters. In many real-world s…
Bayesian OptimizationHyperparameter Optimization