State-Conditioned Adversarial Subgoal Generation

2022-01-24 · Vivienne Huiling Wang, Joni Pajarinen, Tinghuai Wang, Joni-Kristian Kämäräinen

Hierarchical reinforcement learning (HRL) proposes to solve difficult tasks by performing decision-making and control at successively higher levels of temporal abstraction. However, off-policy HRL often suffers from the problem of a non-stationary high-level policy since the low-level policy is constantly changing. In this paper, we propose a novel HRL approach for mitigating the non-stationarity by adversarially enforcing the high-level policy to generate subgoals compatible with the current instantiation of the low-level policy. In practice, the adversarial learning is implemented by training a simple state-conditioned discriminator network concurrently with the high-level policy which determines the compatibility level of subgoals. Comparison to state-of-the-art algorithms shows that our approach improves both learning efficiency and performance in challenging continuous control tasks.

📄 PDF Abstract BibTeX arXiv:2201.09635

Code (0)

등록된 구현이 없습니다.

Tasks

continuous-control Continuous Control Decision Making Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

Anticipation-VLA: Solving Long-Horizon Embodied Tasks via Anticipation-based Subgoal Generation

2026-05-03 · Zhilong Zhang, Wenyu Luo, Haonan Wang, Yifei Sheng 외 arxiv

Vision-Language-Action (VLA) models have emerged as a powerful paradigm for embodied intelligence, enabling robots to perform tasks based on natural language instructions and current visual input. However, existing VLA m…

Learning Subgoal Representations with Slow Dynamics

2021-01-01 · ICLR 2021 1 · Siyuan Li, Lulu Zheng, Jianhao Wang, Chongjie Zhang

In goal-conditioned Hierarchical Reinforcement Learning (HRL), a high-level policy periodically sets subgoals for a low-level policy, and the low-level policy is trained to reach those subgoals. A proper subgoal represen…

continuous-control Continuous Control Hierarchical Reinforcement Learning Representation Learning

Imitating Graph-Based Planning with Goal-Conditioned Policies

2023-03-20 · Junsu Kim, Younggyo Seo, Sungsoo Ahn, Kyunghwan Son 외

Recently, graph-based planning algorithms have gained much attention to solve goal-conditioned reinforcement learning (RL) tasks: they provide a sequence of subgoals to reach the target-goal, and the agents learn to exec…

Reinforcement Learning (RL)

Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generation

2019-09-12 · ICLR 2020 1 · Suraj Nair, Chelsea Finn

Video prediction models combined with planning algorithms have shown promise in enabling robots to learn to perform many vision-based tasks through only self-supervision, reaching novel goals in cluttered scenes with uns…

Self-Supervised Learning Video Prediction

Advancing Autonomous VLM Agents via Variational Subgoal-Conditioned Reinforcement Learning

2025-02-11 · Qingyuan Wu, Jianheng Liu, Jianye Hao, Jun Wang 외

State-of-the-art (SOTA) reinforcement learning (RL) methods have enabled vision-language model (VLM) agents to learn from interaction with online environments without human supervision. However, these methods often strug…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1