Reward Shaping via Diffusion Process in Reinforcement Learning

2023-06-20 · Peeyush Kumar

Reinforcement Learning (RL) models have continually evolved to navigate the exploration - exploitation trade-off in uncertain Markov Decision Processes (MDPs). In this study, I leverage the principles of stochastic thermodynamics and system dynamics to explore reward shaping via diffusion processes. This provides an elegant framework as a way to think about exploration-exploitation trade-off. This article sheds light on relationships between information entropy, stochastic system dynamics, and their influences on entropy production. This exploration allows us to construct a dual-pronged framework that can be interpreted as either a maximum entropy program for deriving efficient policies or a modified cost optimization program accounting for informational costs and benefits. This work presents a novel perspective on the physical nature of information and its implications for online learning in MDPs, consequently providing a better understanding of information-oriented formulations in RL.

📄 PDF Abstract BibTeX arXiv:2306.11885

Code (0)

등록된 구현이 없습니다.

Tasks

Navigatereinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Temporal-Logic-Based Reward Shaping for Continuing Reinforcement Learning Tasks

2020-07-03 · Yuqian Jiang, Sudarshanan Bharadwaj, Bo Wu, Rishi Shah 외

In continuing tasks, average-reward reinforcement learning may be a more appropriate problem formulation than the more common discounted reward formulation. As usual, learning an optimal policy in this setting typically …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

A new Potential-Based Reward Shaping for Reinforcement Learning Agent

2019-02-17 · Babak Badnava, Mona Esmaeili, Nasser Mozayani, Payman Zarkesh-Ha

Potential-based reward shaping (PBRS) is a particular category of machine learning methods which aims to improve the learning speed of a reinforcement learning agent by extracting and utilizing extra knowledge while perf…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applications

2024-07-22 · Sinan Ibrahim, Mostafa Mostafa, Ali Jnadi, Hadi Salloum 외

The aim of Reinforcement Learning (RL) in real-world applications is to create systems capable of making autonomous decisions by learning from their environment through trial and error. This paper emphasizes the importan…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning to Shape Rewards using a Game of Two Partners

2021-03-16 · David Mguni, Taher Jafferjee, Jianhong Wang, Nicolas Perez-Nieves 외

Reward shaping (RS) is a powerful method in reinforcement learning (RL) for overcoming the problem of sparse or uninformative rewards. However, RS typically relies on manually engineered shaping-reward functions whose co…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Vocal Bursts Valence Prediction

Learning to Dynamically Select Between Reward Shaping Signals

2021-01-01 · Alexander Politowicz, Bing Liu

Reinforcement learning (RL) algorithms often have the limitation of sample complexity. Previous research has shown that the reliance on large amounts of experience can be mitigated through the presence of additional feed…

Reinforcement Learning (RL)