Robust Bayesian Optimization with Student-t Likelihood
Bayesian optimization has recently attracted the attention of the automatic machine learning community for its excellent results in hyperparameter tuning. BO is characterized by the sample efficiency with which it can optimize expensive black-box functions. The efficiency is achieved in a similar fashion to the learning to learn methods: surrogate models (typically in the form of Gaussian processes) learn the target function and perform intelligent sampling. This surrogate model can be applied even in the presence of noise; however, as with most regression methods, it is very sensitive to outlier data. This can result in erroneous predictions and, in the case of BO, biased and inefficient exploration. In this work, we present a GP model that is robust to outliers which uses a Student-t likelihood to segregate outliers and robustly conduct Bayesian optimization. We present numerical results evaluating the proposed method in both artificial functions and real problems.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian OptimizationGaussian ProcessesSimilar Papers 제목 키워드 기반
Practical Bayesian optimization in the presence of outliers
Inference in the presence of outliers is an important field of research as outliers are ubiquitous and may arise across a variety of problems and domains. Bayesian optimization is method that heavily relies on probabilis…
Bayesian OptimizationregressionRethinking Likelihood distributions: Student's t Likelihood Boosts Bayesian Neural Network Performance
In Bayesian neural networks (BNNs), variational inference is a widely adopted framework for modeling uncertainty in a distributional way, with the evidence lower bound (ELBO) serving as the standard objective function. S…
Student-t processes as infinite-width limits of posterior Bayesian neural networks
The asymptotic properties of Bayesian Neural Networks (BNNs) have been extensively studied, particularly regarding their approximations by Gaussian processes in the infinite-width limit. We extend these results by showin…
Gaussian ProcessesBayesian linear regression with Student-t assumptions
As an automatic method of determining model complexity using the training data alone, Bayesian linear regression provides us a principled way to select hyperparameters. But one often needs approximation inference if dist…
regressionHypervolume-based Multi-objective Bayesian Optimization with Student-t Processes
Student-$t$ processes have recently been proposed as an appealing alternative non-parameteric function prior. They feature enhanced flexibility and predictive variance. In this work the use of Student-$t$ processes are e…
Bayesian OptimizationGaussian Processes