Deep Reinforcement Learning With Macro-Actions

2016-06-15 · Ishan P. Durugkar, Clemens Rosenbaum, Stefan Dernbach, Sridhar Mahadevan

Deep reinforcement learning has been shown to be a powerful framework for learning policies from complex high-dimensional sensory inputs to actions in complex tasks, such as the Atari domain. In this paper, we explore output representation modeling in the form of temporal abstraction to improve convergence and reliability of deep reinforcement learning approaches. We concentrate on macro-actions, and evaluate these on different Atari 2600 games, where we show that they yield significant improvements in learning speed. Additionally, we show that they can even achieve better scores than DQN. We offer analysis and explanation for both convergence and final results, revealing a problem deep RL approaches have with sparse reward signals.

📄 PDF Abstract BibTeX arXiv:1606.04615

Code (0)

등록된 구현이 없습니다.

Tasks

Atari GamesDeep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

DQN A DQN, or Deep Q-Network, approximates a state-value function in a Q-Learning framework with a neural network. In the Atari…

Macro Action Reinforcement Learning with Sequence Disentanglement using Variational Autoencoder

2019-03-22 · Heecheol Kim, Masanori Yamada, Kosuke Miyoshi, Hiroshi Yamakawa

One problem in the application of reinforcement learning to real-world problems is the curse of dimensionality on the action space. Macro actions, a sequence of primitive actions, have been studied to diminish the dimens…

Disentanglement General Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Reusability and Transferability of Macro Actions for Reinforcement Learning

2019-08-05 · Yi-Hsiang Chang, Kuan-Yu Chang, Henry Kuo, Chun-Yi Lee

Conventional reinforcement learning (RL) typically determines an appropriate primitive action at each timestep. However, by using a proper macro action, defined as a sequence of primitive actions, an agent is able to byp…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Macro-Action-Based Deep Multi-Agent Reinforcement Learning

2020-04-18 · Yuchen Xiao, Joshua Hoffman, Christopher Amato

In real-world multi-robot systems, performing high-quality, collaborative behaviors requires robots to asynchronously reason about high-level action selection at varying time durations. Macro-Action Decentralized Partial…

Decision Making Decision Making Under Uncertainty Deep Reinforcement Learning Multi-agent Reinforcement Learning+3

Hierarchical Meta-Reinforcement Learning via Automated Macro-Action Discovery

2024-12-16 · Minjae Cho, Chuangchuang Sun

Meta-Reinforcement Learning (Meta-RL) enables fast adaptation to new testing tasks. Despite recent advancements, it is still challenging to learn performant policies across multiple complex and high-dimensional tasks. To…

Meta Reinforcement Learning reinforcement-learning Reinforcement Learning

Meta-learning how to Share Credit among Macro-Actions

2025-06-16 · Ionel-Alexandru Hosu, Traian Rebedea, Razvan Pascanu

One proposed mechanism to improve exploration in reinforcement learning is through the use of macro-actions. Paradoxically though, in many scenarios the naive addition of macro-actions does not lead to better exploration…

Atari Games Meta-Learning