Deep Multi-Agent Reinforcement Learning with Discrete-Continuous Hybrid Action Spaces

2019-03-12 · Haotian Fu, Hongyao Tang, Jianye Hao, Zihan Lei, Yingfeng Chen, Changjie Fan

Deep Reinforcement Learning (DRL) has been applied to address a variety of cooperative multi-agent problems with either discrete action spaces or continuous action spaces. However, to the best of our knowledge, no previous work has ever succeeded in applying DRL to multi-agent problems with discrete-continuous hybrid (or parameterized) action spaces which is very common in practice. Our work fills this gap by proposing two novel algorithms: Deep Multi-Agent Parameterized Q-Networks (Deep MAPQN) and Deep Multi-Agent Hierarchical Hybrid Q-Networks (Deep MAHHQN). We follow the centralized training but decentralized execution paradigm: different levels of communication between different agents are used to facilitate the training process, while each agent executes its policy independently based on local observations during execution. Our empirical results on several challenging tasks (simulated RoboCup Soccer and game Ghost Story) show that both Deep MAPQN and Deep MAHHQN are effective and significantly outperform existing independent deep parameterized Q-learning method.

📄 PDF Abstract BibTeX arXiv:1903.04959

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningMulti-agent Reinforcement LearningQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Deep Multi-Agent Reinforcement Learning with Hybrid Action Spaces based on Maximum Entropy

2022-06-10 · Hongzhi Hua, Kaigui Wu, Guixuan Wen

Multi-agent deep reinforcement learning has been applied to address a variety of complex problems with either discrete or continuous action spaces and achieved great success. However, most real-world environments cannot …

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

A further exploration of deep Multi-Agent Reinforcement Learning with Hybrid Action Space

2022-08-30 · Hongzhi Hua, Guixuan Wen, Kaigui Wu

The research of extending deep reinforcement learning (drl) to multi-agent field has solved many complicated problems and made great achievements. However, almost all these studies only focus on discrete or continuous ac…

Deep Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation

2021-09-12 · ICLR 2022 4 · Boyan Li, Hongyao Tang, Yan Zheng, Jianye Hao 외

Discrete-continuous hybrid action space is a natural setting in many practical problems, such as robot control and game AI. However, most previous Reinforcement Learning (RL) works only demonstrate the success in control…

reinforcement-learning Reinforcement Learning (RL)

Learn Continuously, Act Discretely: Hybrid Action-Space Reinforcement Learning For Optimal Execution

2022-07-22 · Feiyang Pan, Tongzhe Zhang, Ling Luo, Jia He 외

Optimal execution is a sequential decision-making problem for cost-saving in algorithmic trading. Studies have found that reinforcement learning (RL) can help decide the order-splitting sizes. However, a problem remains …

Algorithmic Trading continuous-control Continuous Control Decision Making+3

Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics

2020-01-02 · Michael Neunert, Abbas Abdolmaleki, Markus Wulfmeier, Thomas Lampe 외

Many real-world control problems involve both discrete decision variables - such as the choice of control modes, gear switching or digital outputs - as well as continuous decision variables - such as velocity setpoints, …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)