Efficient Online Bayesian Inference for Neural Bandits
In this paper we present a new algorithm for online (sequential) inference in Bayesian neural networks, and show its suitability for tackling contextual bandit problems. The key idea is to combine the extended Kalman filter (which locally linearizes the likelihood function at each time step) with a (learned or random) low-dimensional affine subspace for the parameters; the use of a subspace enables us to scale our algorithm to models with $\sim 1M$ parameters. While most other neural bandit methods need to store the entire past dataset in order to avoid the problem of "catastrophic forgetting", our approach uses constant memory. This is possible because we represent uncertainty about all the parameters in the model, not just the final linear layer. We show good results on the "Deep Bayesian Bandit Showdown" benchmark, as well as MNIST and a recommender system.
Code (1)
Tasks
Bayesian InferenceRecommendation SystemsSimilar Papers 제목 키워드 기반
Empirical analysis of representation learning and exploration in neural kernel bandits
Neural bandits have been shown to provide an efficient solution to practical sequential decision tasks that have nonlinear reward functions. The main contributor to that success is approximate Bayesian inference, which e…
Bayesian InferenceDecision MakingGaussian ProcessesMulti-Armed Bandits+1BayesOpt: A Bayesian Optimization Library for Nonlinear Optimization, Experimental Design and Bandits
BayesOpt is a library with state-of-the-art Bayesian optimization methods to solve nonlinear optimization, stochastic bandits or sequential experimental design problems. Bayesian optimization is sample efficient by build…
Bayesian OptimizationExperimental DesignHyperparameter OptimizationSequential Monte Carlo Bandits
In this paper we propose a flexible and efficient framework for handling multi-armed bandits, combining sequential Monte Carlo algorithms with hierarchical Bayesian modeling techniques. The framework naturally encompasse…
Multi-Armed BanditsBayesian Bandit Algorithms with Approximate Inference in Stochastic Linear Bandits
Bayesian bandit algorithms with approximate Bayesian inference have been widely used in real-world applications. Despite the superior practical performance, their theoretical justification is less investigated in the lit…
Bayesian InferenceThompson SamplingBayesian Online Model Selection
Online model selection in Bayesian bandits raises a fundamental exploration challenge: When an environment instance is sampled from a prior distribution, how can we design an adaptive strategy that explores multiple band…