Scale-free Adversarial Reinforcement Learning
This paper initiates the study of scale-free learning in Markov Decision Processes (MDPs), where the scale of rewards/losses is unknown to the learner. We design a generic algorithmic framework, \underline{S}cale \underline{C}lipping \underline{B}ound (\texttt{SCB}), and instantiate this framework in both the adversarial Multi-armed Bandit (MAB) setting and the adversarial MDP setting. Through this framework, we achieve the first minimax optimal expected regret bound and the first high-probability regret bound in scale-free adversarial MABs, resolving an open problem raised in \cite{hadiji2023adaptation}. On adversarial MDPs, our framework also give birth to the first scale-free RL algorithm with a $\tilde{\mathcal{O}}(\sqrt{T})$ high-probability regret guarantee.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Model-free Reinforcement Learning in Infinite-horizon Average-reward Markov Decision Processes
Model-free reinforcement learning is known to be memory and computation efficient and more amendable to large scale problems. In this paper, two model-free algorithms are introduced for learning infinite-horizon average-…
Multi-Armed Banditsreinforcement-learningReinforcement LearningReinforcement Learning (RL)Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
Reinforcement learning is well suited for optimizing policies of recommender systems. Current solutions mostly focus on model-free approaches, which require frequent interactions with the real environment, and thus are e…
Generative Adversarial NetworkModel-based Reinforcement LearningRecommendation Systemsreinforcement-learning+2Model-Free $μ$ Synthesis via Adversarial Reinforcement Learning
Motivated by the recent empirical success of policy-based reinforcement learning (RL), there has been a research trend studying the performance of policy-based RL methods on standard control benchmark problems. In this p…
modelreinforcement-learningReinforcement LearningReinforcement Learning (RL)A Model-Based Reinforcement Learning with Adversarial Training for Online Recommendation
Reinforcement learning is effective in optimizing policies for recommender systems. Current solutions mostly focus on model-free approaches, which require frequent interactions with a real environment, and thus are expen…
Generative Adversarial NetworkModel-based Reinforcement LearningRecommendation Systemsreinforcement-learning+2Toward Evaluating Robustness of Deep Reinforcement Learning with Continuous Control
Deep reinforcement learning has achieved great success in many previously difficult reinforcement learning tasks, yet recent studies show that deep RL agents are also unavoidably susceptible to adversarial perturbations,…
continuous-controlContinuous ControlDeep Reinforcement LearningMuJoCo+3