Cascading Reinforcement Learning
Cascading bandits have gained popularity in recent years due to their applicability to recommendation systems and online advertising. In the cascading bandit model, at each timestep, an agent recommends an ordered subset of items (called an item list) from a pool of items, each associated with an unknown attraction probability. Then, the user examines the list, and clicks the first attractive item (if any), and after that, the agent receives a reward. The goal of the agent is to maximize the expected cumulative reward. However, the prior literature on cascading bandits ignores the influences of user states (e.g., historical behaviors) on recommendations and the change of states as the session proceeds. Motivated by this fact, we propose a generalized cascading RL framework, which considers the impact of user states and state transition into decisions. In cascading RL, we need to select items not only with large attraction probabilities but also leading to good successor states. This imposes a huge computational challenge due to the combinatorial action space. To tackle this challenge, we delve into the properties of value functions, and design an oracle BestPerm to efficiently find the optimal item list. Equipped with BestPerm, we develop two algorithms CascadingVI and CascadingBPI, which are both computationally-efficient and sample-efficient, and provide near-optimal regret and sample complexity guarantees. Furthermore, we present experiments to show the improved computational and sample efficiencies of our algorithms compared to straightforward adaptations of existing RL algorithms in practice.
Code (0)
등록된 구현이 없습니다.
Tasks
Recommendation Systemsreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Deep Reinforcement Learning for Power Grid Multi-Stage Cascading Failure Mitigation
Cascading failures in power grids can lead to grid collapse, causing severe disruptions to social operations and economic activities. In certain cases, multi-stage cascading failures can occur. However, existing cascadin…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningMitigating Multi-Stage Cascading Failure by Reinforcement Learning
This paper proposes a cascading failure mitigation strategy based on Reinforcement Learning (RL) method. Firstly, the principles of RL are introduced. Then, the Multi-Stage Cascading Failure (MSCF) problem is presented a…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Power Grid Cascading Failure Mitigation by Reinforcement Learning
This paper proposes a cascading failure mitigation strategy based on Reinforcement Learning (RL). The motivation of the Multi-Stage Cascading Failure (MSCF) problem and its connection with the challenge of climate change…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Real-Time Cascade Mitigation in Power Systems Using Influence Graph Improved by Reinforcement Learning
Despite high reliability, modern power systems with growing renewable penetration face an increasing risk of cascading outages. Real-time cascade mitigation requires fast, complex operational decisions under uncertainty.…
Decision MakingGroup-wise Reinforcement Feature Generation for Optimal and Explainable Representation Space Reconstruction
Representation (feature) space is an environment where data points are vectorized, distances are computed, patterns are characterized, and geometric structures are embedded. Extracting a good representation space is crit…
Feature Engineeringfeature selectionRepresentation Learning