Dealing with Sparse Rewards Using Graph Neural Networks

2022-03-25 · Matvey Gerasyov, Ilya Makarov

Deep reinforcement learning in partially observable environments is a difficult task in itself, and can be further complicated by a sparse reward signal. Most tasks involving navigation in three-dimensional environments provide the agent with extremely limited information. Typically, the agent receives a visual observation input from the environment and is rewarded once at the end of the episode. A good reward function could substantially improve the convergence of reinforcement learning algorithms for such tasks. The classic approach to increase the density of the reward signal is to augment it with supplementary rewards. This technique is called the reward shaping. In this study, we propose two modifications of one of the recent reward shaping methods based on graph convolutional networks: the first involving advanced aggregation functions, and the second utilizing the attention mechanism. We empirically validate the effectiveness of our solutions for the task of navigation in a 3D environment with sparse rewards. For the solution featuring attention mechanism, we are also able to show that the learned attention is concentrated on edges corresponding to important transitions in 3D environment.

📄 PDF Abstract BibTeX arXiv:2203.13424

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Evolution Guided Generative Flow Networks

2024-02-03 · Zarif Ikram, Ling Pan, Dianbo Liu

Generative Flow Networks (GFlowNets) are a family of probabilistic generative models that learn to sample compositional objects proportional to their rewards. One big challenge of GFlowNets is training them effectively w…

Evolutionary Algorithms

Dealing with Sparse Rewards in Reinforcement Learning

2019-10-21 · Joshua Hare

Successfully navigating a complex environment to obtain a desired outcome is a difficult task, that up to recently was believed to be capable only by humans. This perception has been broken down over time, especially wit…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Dealing with Sparse Rewards in Continuous Control Robotics via Heavy-Tailed Policies

2022-06-12 · Souradip Chakraborty, Amrit Singh Bedi, Alec Koppel, Pratap Tokekar 외

In this paper, we present a novel Heavy-Tailed Stochastic Policy Gradient (HT-PSG) algorithm to deal with the challenges of sparse rewards in continuous control problems. Sparse reward is common in continuous control rob…

continuous-control Continuous Control MuJoCo OpenAI Gym

Hindsight Experience Replay

2017-07-05 · NeurIPS 2017 12 · Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider 외

Dealing with sparse rewards is one of the biggest challenges in Reinforcement Learning (RL). We present a novel technique called Hindsight Experience Replay which allows sample-efficient learning from rewards which are s…

Reinforcement Learning Reinforcement Learning (RL)

DiSProD: Differentiable Symbolic Propagation of Distributions for Planning

2023-02-03 · Palash Chatterjee, Ashutosh Chapagain, Weizhe Chen, Roni Khardon

The paper introduces DiSProD, an online planner developed for environments with probabilistic transitions in continuous state and action spaces. DiSProD builds a symbolic graph that captures the distribution of future tr…

Navigate