Re-Examining Linear Embeddings for High-Dimensional Bayesian Optimization
Bayesian optimization (BO) is a popular approach to optimize expensive-to-evaluate black-box functions. A significant challenge in BO is to scale to high-dimensional parameter spaces while retaining sample efficiency. A solution considered in existing literature is to embed the high-dimensional space in a lower-dimensional manifold, often via a random linear embedding. In this paper, we identify several crucial issues and misconceptions about the use of linear embeddings for BO. We study the properties of linear embeddings from the literature and show that some of the design choices in current approaches adversely impact their performance. We show empirically that properly addressing these issues significantly improves the efficacy of linear embeddings for BO on a range of problems, including learning a gait policy for robot locomotion.
Code (1)
Tasks
Bayesian OptimizationMisconceptionsVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Adaptive Linear Embedding for Nonstationary High-Dimensional Optimization
Bayesian Optimization (BO) in high-dimensional spaces remains fundamentally limited by the curse of dimensionality and the rigidity of global low-dimensional assumptions. While Random EMbedding Bayesian Optimization (REM…
Bayesian OptimizationHigh-dimensional Bayesian optimization using low-dimensional feature spaces
Bayesian optimization (BO) is a powerful approach for seeking the global optimum of expensive black-box functions and has proven successful for fine tuning hyper-parameters of machine learning models. However, BO is prac…
Bayesian OptimizationDimensionality ReductionVocal Bursts Intensity PredictionActive Learning of Linear Embeddings for Gaussian Processes
We propose an active learning method for discovering low-dimensional structure in high-dimensional Gaussian process (GP) tasks. Such problems are increasingly frequent and important, but have hitherto presented severe pr…
Active LearningBayesian OptimizationGaussian ProcessesregressionKernel Operator-Theoretic Bayesian Filter for Nonlinear Dynamical Systems
Motivated by the surge of interest in Koopman operator theory, we propose a machine-learning alternative based on a functional Bayesian perspective for operator-theoretic modeling of unknown, data-driven, nonlinear dynam…
Nonlinear Dimensionality Reduction Techniques for Bayesian Optimization
Bayesian optimisation (BO) is a standard approach for sample-efficient global optimisation of expensive black-box functions, yet its scalability to high dimensions remains challenging. Here, we investigate nonlinear dime…
Dimensionality Reduction