Action Space Shaping in Deep Reinforcement Learning

2020-04-02 · Anssi Kanervisto, Christian Scheller, Ville Hautamäki

Reinforcement learning (RL) has been successful in training agents in various learning environments, including video-games. However, such work modifies and shrinks the action space from the game's original. This is to avoid trying "pointless" actions and to ease the implementation. Currently, this is mostly done based on intuition, with little systematic research supporting the design decisions. In this work, we aim to gain insight on these action space modifications by conducting extensive experiments in video-game environments. Our results show how domain-specific removal of actions and discretization of continuous actions can be crucial for successful learning. With these insights, we hope to ease the use of RL in new environments, by clarifying what action-spaces are easy to learn.

📄 PDF Abstract BibTeX arXiv:2004.00980

Code (1)

Miffyli/rl-action-space-shaping 공식 구현

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Environment Shaping in Reinforcement Learning using State Abstraction

2020-06-23 · Parameswaran Kamalaruban, Rati Devidze, Volkan Cevher, Adish Singla

One of the central challenges faced by a reinforcement learning (RL) agent is to effectively learn a (near-)optimal policy in environments with large state spaces having sparse and noisy feedback signals. In real-world a…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Efficient Dialogue Policy from Demonstrations through Shaping

2020-07-01 · ACL 2020 6 · Huimin Wang, Baolin Peng, Kam-Fai Wong

Training a task-oriented dialogue agent with reinforcement learning is prohibitively expensive since it requires a large volume of interactions with users. Human demonstrations can be used to accelerate learning progress…

Domain Adaptation

Subgoal-based Reward Shaping to Improve Efficiency in Reinforcement Learning

2021-04-13 · Takato Okudo, Seiji Yamada

Reinforcement learning, which acquires a policy maximizing long-term rewards, has been actively studied. Unfortunately, this learning type is too slow and difficult to use in practical situations because the state-action…

AI Agent reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reward Shaping via Meta-Learning

2019-01-27 · Haosheng Zou, Tongzheng Ren, Dong Yan, Hang Su 외

Reward shaping is one of the most effective methods to tackle the crucial yet challenging problem of credit assignment in Reinforcement Learning (RL). However, designing shaping functions usually requires much expert kno…

Meta-Learning Reinforcement Learning Reinforcement Learning (RL)

Shaping Advice in Deep Reinforcement Learning

2022-02-19 · Baicen Xiao, Bhaskar Ramasubramanian, Radha Poovendran

Reinforcement learning involves agents interacting with an environment to complete tasks. When rewards provided by the environment are sparse, agents may not receive immediate feedback on the quality of actions that they…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1