Leveraging Sequentiality in Reinforcement Learning from a Single Demonstration

2022-11-09 · Alexandre Chenu, Olivier Serris, Olivier Sigaud, Nicolas Perrin-Gilbert

Deep Reinforcement Learning has been successfully applied to learn robotic control. However, the corresponding algorithms struggle when applied to problems where the agent is only rewarded after achieving a complex task. In this context, using demonstrations can significantly speed up the learning process, but demonstrations can be costly to acquire. In this paper, we propose to leverage a sequential bias to learn control policies for complex robotic tasks using a single demonstration. To do so, our method learns a goal-conditioned policy to control a system between successive low-dimensional goals. This sequential goal-reaching approach raises a problem of compatibility between successive goals: we need to ensure that the state resulting from reaching a goal is compatible with the achievement of the following goals. To tackle this problem, we present a new algorithm called DCIL-II. We show that DCIL-II can solve with unprecedented sample efficiency some challenging simulated tasks such as humanoid locomotion and stand-up as well as fast running with a simulated Cassie robot. Our method leveraging sequentiality is a step towards the resolution of complex robotic tasks under minimal specification effort, a key feature for the next generation of autonomous robots.

📄 PDF Abstract BibTeX arXiv:2211.04786

Code (1)

AlexandreChenu/DCIL_XPAG 공식 구현 jax

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Imagined versus Remembered Stories: Quantifying Differences in Narrative Flow

2022-01-07 · Maarten Sap, Anna Jafarpour, Yejin Choi, Noah A. Smith 외

Lifelong experiences and learned knowledge lead to shared expectations about how common situations tend to unfold. Such knowledge of narrative event flow enables people to weave together a story. However, comparable comp…

Language Modelling Large Language Model Sentence Text Generation

Beyond Joint Demonstrations: Personalized Expert Guidance for Efficient Multi-Agent Reinforcement Learning

2024-03-13 · Peihong Yu, Manav Mishra, Alec Koppel, Carl Busart 외

Multi-Agent Reinforcement Learning (MARL) algorithms face the challenge of efficient exploration due to the exponential increase in the size of the joint state-action space. While demonstration-guided learning has proven…

Efficient Exploration Multi-agent Reinforcement Learning Starcraft

Augmenting Policy Learning with Routines Discovered from a Single Demonstration

2020-12-23 · Zelin Zhao, Chuang Gan, Jiajun Wu, Xiaoxiao Guo 외

Humans can abstract prior knowledge from very little data and use it to boost skill learning. In this paper, we propose routine-augmented policy learning (RAPL), which discovers routines composed of primitive actions fro…

Atari Games Imitation Learning reinforcement-learning Reinforcement Learning+1

One-Shot Learning of Multi-Step Tasks from Observation via Activity Localization in Auxiliary Video

2018-06-29 · Wonjoon Goo, Scott Niekum

Due to burdensome data requirements, learning from demonstration often falls short of its promise to allow users to quickly and naturally program robots. Demonstrations are inherently ambiguous and incomplete, making cor…

One-Shot Learning Reinforcement Learning Task 2

Context is Enough: Empirical Validation of $\textit{Sequentiality}$ on Essays

2025-11-12 · Amal Sunny, Advay Gupta, Vishnu Sreekumar arxiv

Recent work has proposed using Large Language Models (LLMs) to quantify narrative flow through a measure called sequentiality, which combines topic and contextual terms. A recent critique argued that the original results…

Automated Essay Scoring