Sparse Bayesian Optimization
Bayesian optimization (BO) is a powerful approach to sample-efficient optimization of black-box objective functions. However, the application of BO to areas such as recommendation systems often requires taking the interpretability and simplicity of the configurations into consideration, a setting that has not been previously studied in the BO literature. To make BO useful for this setting, we present several regularization-based approaches that allow us to discover sparse and more interpretable configurations. We propose a novel differentiable relaxation based on homotopy continuation that makes it possible to target sparsity by working directly with $L_0$ regularization. We identify failure modes for regularized BO and develop a hyperparameter-free method, sparsity exploring Bayesian optimization (SEBO) that seeks to simultaneously maximize a target objective and sparsity. SEBO and methods based on fixed regularization are evaluated on synthetic and real-world problems, and we show that we are able to efficiently optimize for sparsity.
Code (1)
Tasks
Bayesian OptimizationRecommendation SystemsSimilar Papers 제목 키워드 기반
Sparse Spectrum Gaussian Process for Bayesian Optimization
We propose a novel sparse spectrum approximation of Gaussian process (GP) tailored for Bayesian optimization. Whilst the current sparse spectrum methods provide desired approximations for regression problems, it is obser…
Bayesian OptimisationBayesian OptimizationThompson SamplingScalable Bayesian Optimization with Sparse Gaussian Process Models
This thesis focuses on Bayesian optimization with the improvements coming from two aspects:(i) the use of derivative information to accelerate the optimization convergence; and (ii) the consideration of scalable GPs for …
Bayesian OptimizationFast post-process Bayesian inference with Variational Sparse Bayesian Quadrature
In applied Bayesian inference scenarios, users may have access to a large number of pre-existing model evaluations, for example from maximum-a-posteriori (MAP) optimization runs. However, traditional approximate inferenc…
Active LearningBayesian InferenceVariational InferenceA Stochastic Variance-Reduced Coordinate Descent Algorithm for Learning Sparse Bayesian Network from Discrete High-Dimensional Data
This paper addresses the problem of learning a sparse structure Bayesian network from high-dimensional discrete data. Compared to continuous Bayesian networks, learning a discrete Bayesian network is a challenging proble…
Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes
Bayesian optimization is an effective technique for black-box optimization, but its applicability is typically limited to low-dimensional and small-budget problems due to the cubic complexity of computing the Gaussian pr…
Bayesian OptimizationGaussian Processes