High Dimensional Bayesian Optimisation and Bandits via Additive Models
Bayesian Optimisation (BO) is a technique used in optimising a $D$-dimensional function which is typically expensive to evaluate. While there have been many successes for BO in low dimensions, scaling it to high dimensions has been notoriously difficult. Existing literature on the topic are under very restrictive settings. In this paper, we identify two key challenges in this endeavour. We tackle these challenges by assuming an additive structure for the function. This setting is substantially more expressive and contains a richer class of functions than previous work. We prove that, for additive functions the regret has only linear dependence on $D$ even though the function depends on all $D$ dimensions. We also demonstrate several other statistical and computational benefits in our framework. Via synthetic examples, a scientific simulation and a face detection problem we demonstrate that our method outperforms naive BO on additive functions and on several examples where the function is not additive.
Code (0)
등록된 구현이 없습니다.
Tasks
Additive modelsBayesian OptimisationFace DetectionVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Adaptive, Robust and Scalable Bayesian Filtering for Online Learning
In this thesis, we introduce Bayesian filtering as a principled framework for tackling diverse sequential machine learning problems, including online (continual) learning, prequential (one-step-ahead) forecasting, and co…
Continual LearningMulti-Armed BanditsBayesian Optimisation over Multiple Continuous and Categorical Inputs
Efficient optimisation of black-box problems that comprise both continuous and categorical inputs is important, yet poses significant challenges. We propose a new approach, Continuous and Categorical Bayesian Optimisatio…
Bayesian OptimisationDiversityMulti-Armed BanditsHigh-Dimensional Bayesian Optimisation with Large-Scale Constraints -- An Application to Aeroelastic Tailoring
Design optimisation potentially leads to lightweight aircraft structures with lower environmental impact. Due to the high number of design variables and constraints, these problems are ordinarily solved using gradient-ba…
Bayesian OptimisationDimensionality ReductionContextual Causal Bayesian Optimisation
Causal Bayesian optimisation (CaBO) combines causality with Bayesian optimisation (BO) and shows that there are situations where the optimal reward is not achievable if causal knowledge is ignored. While CaBO exploits ca…
Bayesian OptimisationMulti-Armed BanditsThink Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces
High-dimensional black-box optimisation remains an important yet notoriously challenging problem. Despite the success of Bayesian optimisation methods on continuous domains, domains that are categorical, or that mix cont…
Bayesian Optimisation