Hindsight Reward Tweaking via Conditional Deep Reinforcement Learning

2021-09-06 · Ning Wei, Jiahua Liang, Di Xie, ShiLiang Pu

Designing optimal reward functions has been desired but extremely difficult in reinforcement learning (RL). When it comes to modern complex tasks, sophisticated reward functions are widely used to simplify policy learning yet even a tiny adjustment on them is expensive to evaluate due to the drastically increasing cost of training. To this end, we propose a hindsight reward tweaking approach by designing a novel paradigm for deep reinforcement learning to model the influences of reward functions within a near-optimal space. We simply extend the input observation with a condition vector linearly correlated with the effective environment reward parameters and train the model in a conventional manner except for randomizing reward configurations, obtaining a hyper-policy whose characteristics are sensitively regulated over the condition space. We demonstrate the feasibility of this approach and study one of its potential application in policy performance boosting with multiple MuJoCo tasks.

📄 PDF Abstract BibTeX arXiv:2109.02332

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningMuJoCoreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Reward Tweaking: Maximizing the Total Reward While Planning for Short Horizons

2020-02-09 · Chen Tessler, Shie Mannor

In reinforcement learning, the discount factor $\gamma$ controls the agent's effective planning horizon. Traditionally, this parameter was considered part of the MDP; however, as deep reinforcement learning algorithms te…

continuous-control Continuous Control Deep Reinforcement Learning reinforcement-learning+2

Hindsight policy gradients

2017-11-16 · ICLR 2019 5 · Paulo Rauber, Avinash Ummadisingu, Filipe Mutz, Juergen Schmidhuber

A reinforcement learning agent that needs to pursue different goals across episodes requires a goal-conditional policy. In addition to their potential to generalize desirable behavior to unseen goals, such policies may a…

Policy Gradient Methods reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Where Hindsight Credit Can Reside: A Signed-Capacity View of Token Updates in RLVR

2026-04-13 · Yuhang He, Haodong Wu, Siyi Liu, Hongyu Ge 외 arxiv

Reinforcement Learning with Verifiable Rewards (RLVR) improves the reasoning ability of Large Language Models (LLMs), but sparse outcome rewards make token-level credit assignment difficult. We study token-level credit a…

Reinforcement Learning Mathematical Reasoning

Improvements on Hindsight Learning

2018-09-16 · Ameet Deshpande, Srikanth Sarma, Ashutosh Jha, Balaraman Ravindran

Sparse reward problems are one of the biggest challenges in Reinforcement Learning. Goal-directed tasks are one such sparse reward problems where a reward signal is received only when the goal is reached. One promising w…

Policy Gradient Methods reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Curiosity-Driven Multi-Criteria Hindsight Experience Replay

2019-06-09 · John B. Lanier, Stephen Mcaleer, Pierre Baldi

Dealing with sparse rewards is a longstanding challenge in reinforcement learning. The recent use of hindsight methods have achieved success on a variety of sparse-reward tasks, but they fail on complex tasks such as sta…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)