Outcome-directed Reinforcement Learning by Uncertainty & Temporal Distance-Aware Curriculum Goal Generation

2023-01-27 · Daesol Cho, Seungjae Lee, H. Jin Kim

Current reinforcement learning (RL) often suffers when solving a challenging exploration problem where the desired outcomes or high rewards are rarely observed. Even though curriculum RL, a framework that solves complex tasks by proposing a sequence of surrogate tasks, shows reasonable results, most of the previous works still have difficulty in proposing curriculum due to the absence of a mechanism for obtaining calibrated guidance to the desired outcome state without any prior domain knowledge. To alleviate it, we propose an uncertainty & temporal distance-aware curriculum goal generation method for the outcome-directed RL via solving a bipartite matching problem. It could not only provide precisely calibrated guidance of the curriculum to the desired outcome states but also bring much better sample efficiency and geometry-agnostic curriculum goal proposal capability compared to previous curriculum RL methods. We demonstrate that our algorithm significantly outperforms these prior methods in a variety of challenging navigation tasks and robotic manipulation tasks in a quantitative and qualitative way.

📄 PDF Abstract BibTeX arXiv:2301.11741

Code (1)

jaylee0301/outpace_official 공식 구현 pytorch

Tasks

reinforcement-learning Reinforcement Learning (RL)

MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learning

2021-07-15 · Kevin Li, Abhishek Gupta, Ashwin Reddy, Vitchyr Pong 외

Exploration in reinforcement learning is a challenging problem: in the worst case, the agent must search for high-reward states that could be hidden anywhere in the state space. Can we define a more tractable class of RL…

Meta-Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Directed evolution algorithm drives neural prediction

2025-12-01 · Yanlin Wang, Nancy M Young, Patrick C M Wong arxiv

Neural prediction offers a promising approach to forecasting the individual variability of neurocognitive functions and disorders and providing prognostic indicators for personalized invention. However, it is challenging…

Reinforcement Learning

Directed Exploration for Reinforcement Learning

2019-06-18 · Zhaohan Daniel Guo, Emma Brunskill

Efficient exploration is necessary to achieve good sample efficiency for reinforcement learning in general. From small, tabular settings such as gridworlds to large, continuous and sparse reward settings such as robotic …

Efficient Exploration reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

GroundControl: Anticipating Navigation Failures in Vision-Language Agents via Trajectory-Consistent Uncertainty Estimates

2026-06-18 · Nastaran Darabi, Divake Kumar, Sina Tayebati, Devashri Naik 외 arxiv

Vision-language navigation agents achieve competitive average success on benchmark tasks, yet failures often arise through predictable trajectory-level breakdowns such as oscillation, stagnation, or inefficient detours. …

Vision-Language Navigation

Delay-Empowered Causal Hierarchical Reinforcement Learning

2026-05-12 · Chenran Zhao, Dianxi Shi, Haotian Wang, Mengzhu Wang 외 arxiv

Many real-world tasks involve delayed effects, where the outcomes of actions emerge after varying time lags. Existing delay-aware reinforcement learning methods often rely on state augmentation, prior knowledge of delay …

Hierarchical Reinforcement Learning