Global optimization of Lipschitz functions
The goal of the paper is to design sequential strategies which lead to efficient optimization of an unknown function under the only assumption that it has a finite Lipschitz constant. We first identify sufficient conditions for the consistency of generic sequential algorithms and formulate the expected minimax rate for their performance. We introduce and analyze a first algorithm called LIPO which assumes the Lipschitz constant to be known. Consistency, minimax rates for LIPO are proved, as well as fast rates under an additional H\"older like condition. An adaptive version of LIPO is also introduced for the more realistic setup where the Lipschitz constant is unknown and has to be estimated along with the optimization. Similar theoretical guarantees are shown to hold for the adaptive LIPO algorithm and a numerical assessment is provided at the end of the paper to illustrate the potential of this strategy with respect to state-of-the-art methods over typical benchmark problems for global optimization.
Code (3)
Tasks
global-optimizationHyperparameter OptimizationSimilar Papers 제목 키워드 기반
Every Call is Precious: Global Optimization of Black-Box Functions with Unknown Lipschitz Constants
Optimizing expensive, non-convex, black-box Lipschitz continuous functions presents significant challenges, particularly when the Lipschitz constant of the underlying function is unknown. Such problems often demand numer…
global-optimizationCumulative Regret Analysis of the Piyavskii--Shubert Algorithm and Its Variants for Global Optimization
We study the problem of global optimization, where we analyze the performance of the Piyavskii--Shubert algorithm and its variants. For any given time duration $T$, instead of the extensively studied simple regret (which…
global-optimizationLow Regret Binary Sampling Method for Efficient Global Optimization of Univariate Functions
In this work, we propose a computationally efficient algorithm for the problem of global optimization in univariate loss functions. For the performance evaluation, we study the cumulative regret of the algorithm instead …
global-optimizationEfficient Minimax Optimal Global Optimization of Lipschitz Continuous Multivariate Functions
In this work, we propose an efficient minimax optimal global optimization algorithm for multivariate Lipschitz continuous functions. To evaluate the performance of our approach, we utilize the average regret instead of t…
global-optimizationCombining Bayesian Optimization and Lipschitz Optimization
Bayesian optimization and Lipschitz optimization have developed alternative techniques for optimizing black-box functions. They each exploit a different form of prior about the function. In this work, we explore strategi…
Bayesian Optimizationglobal-optimizationThompson Sampling