Reward Shaping Using Convolutional Neural Network

2022-10-30 · Hani Sami, Hadi Otrok, Jamal Bentahar, Azzam Mourad, Ernesto Damiani

In this paper, we propose Value Iteration Network for Reward Shaping (VIN-RS), a potential-based reward shaping mechanism using Convolutional Neural Network (CNN). The proposed VIN-RS embeds a CNN trained on computed labels using the message passing mechanism of the Hidden Markov Model. The CNN processes images or graphs of the environment to predict the shaping values. Recent work on reward shaping still has limitations towards training on a representation of the Markov Decision Process (MDP) and building an estimate of the transition matrix. The advantage of VIN-RS is to construct an effective potential function from an estimated MDP while automatically inferring the environment transition matrix. The proposed VIN-RS estimates the transition matrix through a self-learned convolution filter while extracting environment details from the input frames or sampled graphs. Due to (1) the previous success of using message passing for reward shaping; and (2) the CNN planning behavior, we use these messages to train the CNN of VIN-RS. Experiments are performed on tabular games, Atari 2600 and MuJoCo, for discrete and continuous action space. Our results illustrate promising improvements in the learning speed and maximum cumulative reward compared to the state-of-the-art.

📄 PDF Abstract BibTeX arXiv:2210.16956

Code (0)

등록된 구현이 없습니다.

Tasks

MuJoCo

Methods 이 논문이 사용한 방법론

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Reward Propagation Using Graph Convolutional Networks

2020-10-06 · NeurIPS 2020 12 · Martin Klissarov, Doina Precup

Potential-based reward shaping provides an approach for designing good reward functions, with the purpose of speeding up learning. However, automatically finding potential functions for complex environments is a difficul…

Graph Representation Learning Representation Learning

Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping

2020-11-05 · NeurIPS 2020 12 · Yujing Hu, Weixun Wang, Hangtian Jia, Yixiang Wang 외

Reward shaping is an effective technique for incorporating domain knowledge into reinforcement learning (RL). Existing approaches such as potential-based reward shaping normally make full use of a given shaping reward fu…

MuJoCo Reinforcement Learning (RL)

Learning to Shape Rewards using a Game of Two Partners

2021-03-16 · David Mguni, Taher Jafferjee, Jianhong Wang, Nicolas Perez-Nieves 외

Reward shaping (RS) is a powerful method in reinforcement learning (RL) for overcoming the problem of sparse or uninformative rewards. However, RS typically relies on manually engineered shaping-reward functions whose co…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Valence Prediction

Learning to Dynamically Select Between Reward Shaping Signals

2021-01-01 · Alexander Politowicz, Bing Liu

Reinforcement learning (RL) algorithms often have the limitation of sample complexity. Previous research has shown that the reliance on large amounts of experience can be mitigated through the presence of additional feed…

Reinforcement Learning (RL)

Dealing with Sparse Rewards Using Graph Neural Networks

2022-03-25 · Matvey Gerasyov, Ilya Makarov

Deep reinforcement learning in partially observable environments is a difficult task in itself, and can be further complicated by a sparse reward signal. Most tasks involving navigation in three-dimensional environments …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)