Reinforcement Learning with Goal-Distance Gradient

2020-01-01 · Kai Jiang, XiaoLong Qin

Reinforcement learning usually uses the feedback rewards of environmental to train agents. But the rewards in the actual environment are sparse, and even some environments will not rewards. Most of the current methods are difficult to get good performance in sparse reward or non-reward environments. Although using shaped rewards is effective when solving sparse reward tasks, it is limited to specific problems and learning is also susceptible to local optima. We propose a model-free method that does not rely on environmental rewards to solve the problem of sparse rewards in the general environment. Our method use the minimum number of transitions between states as the distance to replace the rewards of environmental, and proposes a goal-distance gradient to achieve policy improvement. We also introduce a bridge point planning method based on the characteristics of our method to improve exploration efficiency, thereby solving more complex tasks. Experiments show that our method performs better on sparse reward and local optimal problems in complex environments than previous work.

📄 PDF Abstract BibTeX arXiv:2001.00127

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Dynamical Distance Learning for Semi-Supervised and Unsupervised Skill Discovery

2019-07-18 · ICLR 2020 1 · Kristian Hartikainen, Xinyang Geng, Tuomas Haarnoja, Sergey Levine

Reinforcement learning requires manual specification of a reward function to learn a task. While in principle this reward function only needs to specify the task goal, in practice reinforcement learning can be very time-…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Universal Planning Networks

2018-04-02 · Aravind Srinivas, Allan Jabri, Pieter Abbeel, Sergey Levine 외

A key challenge in complex visuomotor control is learning abstract representations that are effective for specifying goals, planning, and generalization. To this end, we introduce universal planning networks (UPN). UPNs …

Imitation Learning Reinforcement Learning Representation Learning Transfer Reinforcement Learning

Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning

2026-06-15 · Swaminathan S K, Damiya Gondha, Theyanesh Eswaramoorthy Rajahkrishnan, Aritra Hazra arxiv

Hamilton-Jacobi-Bellman theory implies that the optimal goal-conditioned action depends on the goal only through the gradient of the goal-reaching distance at the current state, yet standard online GCRL still conditions …

Reinforcement Learning

Self-supervised Learning of Distance Functions for Goal-Conditioned Reinforcement Learning

2019-07-05 · Srinivas Venkattaramanujam, Eric Crawford, Thang Doan, Doina Precup

Goal-conditioned policies are used in order to break down complex reinforcement learning (RL) problems by using subgoals, which can be defined either in state space or in a latent feature space. This can increase the eff…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Self-Supervised Learning

Cramer Type Distances for Learning Gaussian Mixture Models by Gradient Descent

2023-07-13 · Ruichong Zhang

The learning of Gaussian Mixture Models (also referred to simply as GMMs) plays an important role in machine learning. Known for their expressiveness and interpretability, Gaussian mixture models have a wide range of app…

Distributional Reinforcement Learning