Online Limited Memory Neural-Linear Bandits
We study neural-linear bandits for solving problems where both exploration and representation learning play an important role. Neural-linear bandits leverage the representation power of deep neural networks and combine it with efficient exploration mechanisms, designed for linear contextual bandits, on top of the last hidden layer. Since the representation is optimized during learning, information regarding exploration with “old” features is lost. We propose the first limited memory neural- linear bandit that is resilient to this catastrophic forgetting phenomenon by solving a semi-definite program. We then approximate the semi-definite program using stochastic gradient descent to make the algorithm practical and adjusted for online usage. We perform simulations on a variety of data sets, including regression, classification, and sentiment analysis. In addition, we evaluate our algorithm in a challenging uplink rate-control application. The bandit controls the transmission rates of data segments over cellular links to achieve optimal throughput. We observe that our algorithm achieves superior performance and shows resilience to catastrophic forgetting.
Code (0)
등록된 구현이 없습니다.
Tasks
Efficient ExplorationMulti-Armed BanditsRepresentation LearningSentiment AnalysisSimilar Papers 제목 키워드 기반
Online Limited Memory Neural-Linear Bandits with Likelihood Matching
We study neural-linear bandits for solving problems where {\em both} exploration and representation learning play an important role. Neural-linear bandits harnesses the representation power of Deep Neural Networks (DNNs)…
Efficient ExplorationMulti-Armed BanditsRepresentation LearningOnline Learning with Recency: Algorithms for Sliding-window Streaming Multi-armed Bandits
Motivated by the recency effect in online learning, we study algorithms for single-pass *sliding-window streaming multi-armed bandits (MABs)* in this paper. In this setting, we are given $n$ arms with unknown sub-Gaussia…
Multi-Armed BanditsTight Memory-Regret Lower Bounds for Streaming Bandits
In this paper, we investigate the streaming bandits problem, wherein the learner aims to minimize regret by dealing with online arriving arms and sublinear arm memory. We establish the tight worst-case regret lower bound…
Neural Linear Bandits: Overcoming Catastrophic Forgetting through Likelihood Matching
We study neural-linear bandits for solving problems where both exploration and representation learning play an important role. Neural-linear bandits leverage the representation power of deep neural networks and combine i…
Efficient ExplorationMulti-Armed BanditsregressionRepresentation Learning+1Deep Neural Linear Bandits: Overcoming Catastrophic Forgetting through Likelihood Matching
We study the neural-linear bandit model for solving sequential decision-making problems with high dimensional side information. Neural-linear bandits leverage the representation power of deep neural networks and combine …
Decision MakingEfficient ExplorationGeneral ClassificationMulti-Armed Bandits+3