Distributional Reward Decomposition for Reinforcement Learning

2019-11-06 · NeurIPS 2019 12 · Zichuan Lin, Li Zhao, Derek Yang, Tao Qin, Guangwen Yang, Tie-Yan Liu

Many reinforcement learning (RL) tasks have specific properties that can be leveraged to modify existing RL algorithms to adapt to those tasks and further improve performance, and a general class of such properties is the multiple reward channel. In those environments the full reward can be decomposed into sub-rewards obtained from different channels. Existing work on reward decomposition either requires prior knowledge of the environment to decompose the full reward, or decomposes reward without prior knowledge but with degraded performance. In this paper, we propose Distributional Reward Decomposition for Reinforcement Learning (DRDRL), a novel reward decomposition algorithm which captures the multiple reward channel structure under distributional setting. Empirically, our method captures the multi-channel structure and discovers meaningful reward decomposition, without any requirements on prior knowledge. Consequently, our agent achieves better performance than existing methods on environments with multiple reward channels.

📄 PDF Abstract BibTeX arXiv:1911.02166

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Noise Distribution Decomposition based Multi-Agent Distributional Reinforcement Learning

2023-12-12 · Wei Geng, Baidi Xiao, Rongpeng Li, Ning Wei 외

Generally, Reinforcement Learning (RL) agent updates its policy by repetitively interacting with the environment, contingent on the received rewards to observed states and undertaken actions. However, the environmental d…

Distributional Reinforcement Learning Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning+1

A Local Temporal Difference Code for Distributional Reinforcement Learning

2020-12-01 · NeurIPS 2020 12 · Pablo Tano, Peter Dayan, Alexandre Pouget

Recent theoretical and experimental results suggest that the dopamine system implements distributional temporal difference backups, allowing learning of the entire distributions of the long-run values of states rather th…

Distributional Reinforcement Learning Imputation reinforcement-learning Reinforcement Learning+1

Offline Multi-agent Reinforcement Learning via Score Decomposition

2025-05-09 · Dan Qiao, Wenhao Li, Shanchao Yang, Hongyuan Zha 외

Offline cooperative multi-agent reinforcement learning (MARL) faces unique challenges due to distributional shifts, particularly stemming from the high dimensionality of joint action spaces and the presence of out-of-dis…

continuous-control Continuous Control MuJoCo Multi-agent Reinforcement Learning+2

Distributional Reinforcement Learning for Multi-Dimensional Reward Functions

2021-10-26 · NeurIPS 2021 12 · Pushi Zhang, Xiaoyu Chen, Li Zhao, Wei Xiong 외

A growing trend for value-based reinforcement learning (RL) algorithms is to capture more information than scalar value functions in the value network. One of the most well-known methods in this branch is distributional …

Distributional Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

DSAC: Distributional Soft Actor Critic for Risk-Sensitive Reinforcement Learning

2020-04-30 · Xiaoteng Ma, Li Xia, Zhengyuan Zhou, Jun Yang 외

In this paper, we present a new reinforcement learning (RL) algorithm called Distributional Soft Actor Critic (DSAC), which exploits the distributional information of accumulated rewards to achieve better performance. Se…

continuous-control Continuous Control reinforcement-learning Reinforcement Learning+1