Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learning

2018-11-22 · Sainbayar Sukhbaatar, Emily Denton, Arthur Szlam, Rob Fergus

In hierarchical reinforcement learning a major challenge is determining appropriate low-level policies. We propose an unsupervised learning scheme, based on asymmetric self-play from Sukhbaatar et al. (2018), that automatically learns a good representation of sub-goals in the environment and a low-level policy that can execute them. A high-level policy can then direct the lower one by generating a sequence of continuous sub-goal vectors. We evaluate our model using Mazebase and Mujoco environments, including the challenging AntGather task. Visualizations of the sub-goal embeddings reveal a logical decomposition of tasks within the environment. Quantitatively, our approach obtains compelling performance gains over non-hierarchical approaches.

📄 PDF Abstract BibTeX arXiv:1811.09083

Code (2)

ACampero/hsp pytorch

tesatory/hsp pytorch

Tasks

Hierarchical Reinforcement LearningMuJoCoreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Disentangled Skill Embeddings for Reinforcement Learning

2019-06-21 · Janith C. Petangoda, Sergio Pascual-Diaz, Vincent Adam, Peter Vrancx 외

We propose a novel framework for multi-task reinforcement learning (MTRL). Using a variational inference formulation, we learn policies that generalize across both changing dynamics and goals. The resulting policies are …

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Mastering Multi-Drone Volleyball through Hierarchical Co-Self-Play Reinforcement Learning

2025-05-07 · Ruize Zhang, Sirui Xiang, Zelai Xu, Feng Gao 외

In this paper, we tackle the problem of learning to play 3v3 multi-drone volleyball, a new embodied competitive task that requires both high-level strategic coordination and low-level agile control. The task is turn-base…

Hierarchical Reinforcement Learning

Efficient Adaptation in Mixed-Motive Environments via Hierarchical Opponent Modeling and Planning

2024-06-12 · Yizhe Huang, Anji Liu, Fanqi Kong, Yaodong Yang 외

Despite the recent successes of multi-agent reinforcement learning (MARL) algorithms, efficiently adapting to co-players in mixed-motive environments remains a significant challenge. One feasible approach is to hierarchi…

Decision Making Multi-agent Reinforcement Learning

Hierarchical Universal Value Function Approximators

2024-10-11 · Rushiv Arora

There have been key advancements to building universal approximators for multi-goal collections of reinforcement learning value functions -- key elements in estimating long-term returns of states in a parameterized manne…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning

Hierarchical Text Generation and Planning for Strategic Dialogue

2017-12-15 · ICML 2018 7 · Denis Yarats, Mike Lewis

End-to-end models for goal-orientated dialogue are challenging to train, because linguistic and strategic aspects are entangled in latent state vectors. We introduce an approach to learning representations of messages in…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+2