High-Dimensional Bayesian Optimization via Nested Riemannian Manifolds
Despite the recent success of Bayesian optimization (BO) in a variety of applications where sample efficiency is imperative, its performance may be seriously compromised in settings characterized by high-dimensional parameter spaces. A solution to preserve the sample efficiency of BO in such problems is to introduce domain knowledge into its formulation. In this paper, we propose to exploit the geometry of non-Euclidean search spaces, which often arise in a variety of domains, to learn structure-preserving mappings and optimize the acquisition function of BO in low-dimensional latent spaces. Our approach, built on Riemannian manifolds theory, features geometry-aware Gaussian processes that jointly learn a nested-manifold embedding and a representation of the objective function in the latent space. We test our approach in several benchmark artificial landscapes and report that it not only outperforms other high-dimensional BO approaches in several settings, but consistently optimizes the objective functions, as opposed to geometry-unaware BO methods.
Code (1)
Tasks
Bayesian OptimizationGaussian ProcessesVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Nested Grassmannians for Dimensionality Reduction with Applications
In the recent past, nested structures in Riemannian manifolds has been studied in the context of dimensionality reduction as an alternative to the popular principal geodesic analysis (PGA) technique, for example, the pri…
Action RecognitionClusteringDimensionality ReductionFace Recognition+1Riemannian Stochastic Gradient Method for Nested Composition Optimization
This work considers optimization of composition of functions in a nested form over Riemannian manifolds where each function contains an expectation. This type of problems is gaining popularity in applications such as pol…
Meta-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces
Recent advances have extended the scope of Bayesian optimization (BO) to expensive-to-evaluate black-box functions with dozens of dimensions, aspiring to unlock impactful applications, for example, in the life sciences, …
Bayesian OptimizationNeural Architecture SearchBounce: Reliable High-Dimensional Bayesian Optimization for Combinatorial and Mixed Spaces
Impactful applications such as materials discovery, hardware design, neural architecture search, or portfolio optimization require optimizing high-dimensional black-box functions with mixed and combinatorial input spaces…
Bayesian OptimizationNeural Architecture SearchPortfolio OptimizationBayesian Optimization Meets Riemannian Manifolds in Robot Learning
Bayesian optimization (BO) recently became popular in robotics to optimize control parameters and parametric policies in direct reinforcement learning due to its data efficiency and gradient-free approach. However, its p…
Bayesian OptimizationReinforcement Learning