Solving Compositional Reinforcement Learning Problems via Task Reduction

2021-03-13 · ICLR 2021 1 · Yunfei Li, Yilin Wu, Huazhe Xu, Xiaolong Wang, Yi Wu

We propose a novel learning paradigm, Self-Imitation via Reduction (SIR), for solving compositional reinforcement learning problems. SIR is based on two core ideas: task reduction and self-imitation. Task reduction tackles a hard-to-solve task by actively reducing it to an easier task whose solution is known by the RL agent. Once the original hard task is successfully solved by task reduction, the agent naturally obtains a self-generated solution trajectory to imitate. By continuously collecting and imitating such demonstrations, the agent is able to progressively expand the solved subspace in the entire task space. Experiment results show that SIR can significantly accelerate and improve learning on a variety of challenging sparse-reward continuous-control problems with compositional structures. Code and videos are available at https://sites.google.com/view/sir-compositional.

📄 PDF Abstract BibTeX arXiv:2103.07607

Code (1)

IrisLi17/self-imitation-via-reduction 공식 구현

Tasks

continuous-control Continuous Control reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Language as an Abstraction for Hierarchical Deep Reinforcement Learning

2019-06-18 · NeurIPS 2019 12 · Yiding Jiang, Shixiang Gu, Kevin Murphy, Chelsea Finn

Solving complex, temporally-extended tasks is a long-standing problem in reinforcement learning (RL). We hypothesize that one critical element of solving such problems is the notion of compositionality. With the ability …

Deep Reinforcement Learning Instruction Following MuJoCo Object+4

Re-Tuning: Overcoming the Compositionality Limits of Large Language Models with Recursive Tuning

2024-07-05 · Eric Pasewark, Kyle Montgomery, Kefei Duan, Dawn Song 외

We present a new method for large language models to solve compositional tasks. Although they have shown strong performance on traditional language understanding tasks, large language models struggle to solve composition…

GPU

Environment Generation for Zero-Shot Compositional Reinforcement Learning

2022-01-21 · NeurIPS 2021 12 · Izzeddin Gur, Natasha Jaques, Yingjie Miao, Jongwook Choi 외

Many real-world problems are compositional - solving them requires completing interdependent sub-tasks, either in series or in parallel, that can be represented as a dependency graph. Deep reinforcement learning (RL) age…

Deep Reinforcement Learning Navigate reinforcement-learning Reinforcement Learning+1

Reduce, Reuse, Recycle: Categories for Compositional Reinforcement Learning

2024-08-23 · Georgios Bakirtzis, Michail Savvas, Ruihan Zhao, Sandeep Chinchali 외

In reinforcement learning, conducting task composition by forming cohesive, executable sequences from multiple tasks remains challenging. However, the ability to (de)compose tasks is a linchpin in developing robotic syst…

reinforcement-learning Reinforcement Learning

Asynchronous Stochastic Composition Optimization with Variance Reduction

2018-11-15 · Shuheng Shen, Linli Xu, Jingchang Liu, Junliang Guo 외

Composition optimization has drawn a lot of attention in a wide variety of machine learning domains from risk management to reinforcement learning. Existing methods solving the composition optimization problem often work…

Management Reinforcement Learning