Parameterizing Non-Parametric Meta-Reinforcement Learning Tasks via Subtask Decomposition

2023-09-21 · NeurIPS 2023 11

Meta-reinforcement learning (meta-RL) techniques have demonstrated remarkable success in generalizing deep reinforcement learning across a range of tasks. Nevertheless, these methods often struggle to generalize beyond tasks with parametric variations. To overcome this challenge, we propose Subtask Decomposition and Virtual Training (SDVT), a novel meta-RL approach that decomposes each non-parametric task into a collection of elementary subtasks and parameterizes the task based on its decomposition. We employ a Gaussian mixture VAE to meta-learn the decomposition process, enabling the agent to reuse policies acquired from common subtasks. Additionally, we propose a virtual training procedure, specifically designed for non-parametric task variability, which generates hypothetical subtask compositions, thereby enhancing generalization to previously unseen subtask compositions. Our method significantly improves performance on the Meta-World ML-10 and ML-45 benchmarks, surpassing current state-of-the-art techniques.

📄 PDF Abstract BibTeX

Code (1)

suyoung-lee/sdvt 공식 구현 pytorch

Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization

2022-05-25 · Sungryull Sohn, Hyunjae Woo, Jongwook Choi, lyubing qiang 외

We tackle real-world problems with complex structures beyond the pixel-based game or simulator. We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph that defines a…

Hierarchical Reinforcement Learning Meta Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Meta Reinforcement Learning with Autonomous Inference of Subtask Dependencies

2020-01-01 · ICLR 2020 1 · Sungryull Sohn, Hyunjae Woo, Jongwook Choi, Honglak Lee

We propose and address a novel few-shot RL problem, where a task is characterized by a subtask graph which describes a set of subtasks and their dependencies that are unknown to the agent. The agent needs to quickly adap…

Efficient Exploration Meta Reinforcement Learning reinforcement-learning Reinforcement Learning+3

SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning

2018-10-31 · Daoming Lyu, Fangkai Yang, Bo Liu, Steven Gustafson

Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hie…

Decision Making Deep Reinforcement Learning reinforcement-learning Reinforcement Learning+2

Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learning

2017-06-15 · ICML 2017 8 · Junhyuk Oh, Satinder Singh, Honglak Lee, Pushmeet Kohli

As a step towards developing zero-shot task generalization capabilities in reinforcement learning (RL), we introduce a new RL problem where the agent should learn to execute sequences of instructions after learning usefu…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

MGHRL: Meta Goal-generation for Hierarchical Reinforcement Learning

2019-09-30 · Haotian Fu, Hongyao Tang, Jianye Hao, Wulong Liu 외

Most meta reinforcement learning (meta-RL) methods learn to adapt to new tasks by directly optimizing the parameters of policies over primitive action space. Such algorithms work well in tasks with relatively slight diff…

Hierarchical Reinforcement Learning Meta-Learning Meta Reinforcement Learning reinforcement-learning+2