LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery
We introduce LOTUS, a continual imitation learning algorithm that empowers a physical robot to continuously and efficiently learn to solve new manipulation tasks throughout its lifespan. The core idea behind LOTUS is constructing an ever-growing skill library from a sequence of new tasks with a small number of human demonstrations. LOTUS starts with a continual skill discovery process using an open-vocabulary vision model, which extracts skills as recurring patterns presented in unsegmented demonstrations. Continual skill discovery updates existing skills to avoid catastrophic forgetting of previous tasks and adds new skills to solve novel tasks. LOTUS trains a meta-controller that flexibly composes various skills to tackle vision-based manipulation tasks in the lifelong learning process. Our comprehensive experiments show that LOTUS outperforms state-of-the-art baselines by over 11% in success rate, showing its superior knowledge transfer ability compared to prior methods. More results and videos can be found on the project website: https://ut-austin-rpl.github.io/Lotus/.
Code (0)
등록된 구현이 없습니다.
Tasks
Imitation LearningLifelong learningRobot ManipulationTransfer LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policy
Generalizing language-conditioned robotic policies to new tasks remains a significant challenge, hampered by the lack of suitable simulation benchmarks. In this paper, we address this gap by introducing GemBench, a novel…
Motion PlanningRobot ManipulationRobot Manipulation GeneralizationTask PlanningSPECI: Skill Prompts based Hierarchical Continual Imitation Learning for Robot Manipulation
Real-world robot manipulation in dynamic unstructured environments requires lifelong adaptability to evolving objects, scenes and tasks. Traditional imitation learning relies on static training paradigms, which are ill-s…
Action GenerationImitation LearningRobot ManipulationTransfer LearningCRIL: Continual Robot Imitation Learning via Generative and Prediction Model
Imitation learning (IL) algorithms have shown promising results for robots to learn skills from expert demonstrations. However, they need multi-task demonstrations to be provided at once for acquiring diverse skills, whi…
Generative Adversarial NetworkImitation LearningWorld Action Models Enable Continual Imitation Learning with Recurrent Generative Replays
Going beyond predicting robot actions, World Action Models (WAMs) can also generate future visual observations. We build on this generative capability to propose Recurrent Generative Replay (REGEN), a continual imitation…
A Simple Approach to Continual Learning by Transferring Skill Parameters
In order to be effective general purpose machines in real world environments, robots not only will need to adapt their existing manipulation skills to new circumstances, they will need to acquire entirely new skills on-t…
Continual Learning