Two families of indexable partially observable restless bandits and Whittle index computation
We consider the restless bandits with general state space under partial observability with two observational models: first, the state of each bandit is not observable at all, and second, the state of each bandit is observable only if it is chosen. We assume both models satisfy the restart property under which we prove indexability of the models and propose the Whittle index policy as the solution. For the first model, we derive a closed-form expression for the Whittle index. For the second model, we propose an efficient algorithm to compute the Whittle index by exploiting the qualitative properties of the optimal policy. We present detailed numerical experiments for multiple instances of machine maintenance problem. The result indicates that the Whittle index policy outperforms myopic policy and can be close to optimal in different setups.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Simulation Based Algorithms for Markov Decision Processes and Multi-Action Restless Bandits
We consider multi-dimensional Markov decision processes and formulate a long term discounted reward optimization problem. Two simulation based algorithms---Monte Carlo rollout policy and parallel rollout policy are studi…
Lagrangian Relaxation for Multi-Action Partially Observable Restless Bandits: Heuristic Policies and Indexability
Partially observable restless multi-armed bandits have found numerous applications including in recommendation systems, communication systems, public healthcare outreach systems, and in operations research. We study mult…
Recommendation SystemsMulti-Armed BanditsConstrained Restless Bandits for Dynamic Scheduling in Cyber-Physical Systems
This paper studies a class of constrained restless multi-armed bandits (CRMAB). The constraints are in the form of time varying set of actions (set of available arms). This variation can be either stochastic or semi-dete…
Decision MakingDecision Making Under UncertaintyMulti-Armed BanditsSchedulingWhen are Kalman-filter restless bandits indexable?
We study the restless bandit associated with an extremely simple scalar Kalman filter model in discrete time. Under certain assumptions, we prove that the problem is indexable in the sense that the Whittle index is a non…
Indexability and Rollout Policy for Multi-State Partially Observable Restless Bandits
Restless multi-armed bandits with partially observable states has applications in communication systems, age of information and recommendation systems. In this paper, we study multi-state partially observable restless ba…
Multi-Armed BanditsRecommendation Systems