MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations

2022-12-12 · Nicklas Hansen, Yixin Lin, Hao Su, Xiaolong Wang, Vikash Kumar, Aravind Rajeswaran

Poor sample efficiency continues to be the primary challenge for deployment of deep Reinforcement Learning (RL) algorithms for real-world applications, and in particular for visuo-motor control. Model-based RL has the potential to be highly sample efficient by concurrently learning a world model and using synthetic rollouts for planning and policy improvement. However, in practice, sample-efficient learning with model-based RL is bottlenecked by the exploration challenge. In this work, we find that leveraging just a handful of demonstrations can dramatically improve the sample-efficiency of model-based RL. Simply appending demonstrations to the interaction dataset, however, does not suffice. We identify key ingredients for leveraging demonstrations in model learning -- policy pretraining, targeted exploration, and oversampling of demonstration data -- which forms the three phases of our model-based RL framework. We empirically study three complex visuo-motor control domains and find that our method is 150%-250% more successful in completing sparse reward tasks compared to prior approaches in the low data regime (100K interaction steps, 5 demonstrations). Code and videos are available at: https://nicklashansen.github.io/modemrl

📄 PDF Abstract BibTeX arXiv:2212.05698

Code (1)

facebookresearch/modem 공식 구현 pytorch

Tasks

Deep Reinforcement LearningModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation

2023-09-25 · Patrick Lancaster, Nicklas Hansen, Aravind Rajeswaran, Vikash Kumar

Robotic systems that aspire to operate in uninstrumented real-world environments must perceive the world directly via onboard sensing. Vision-based learning systems aim to eliminate the need for environment instrumentati…

Contact-rich Manipulation Model-based Reinforcement Learning Robot Manipulation State Estimation

Accelerating Self-Imitation Learning from Demonstrations via Policy Constraints and Q-Ensemble

2022-12-07 · Chao Li

Deep reinforcement learning (DRL) provides a new way to generate robot control policy. However, the process of training control policy requires lengthy exploration, resulting in a low sample efficiency of reinforcement l…

continuous-control Continuous Control Deep Reinforcement Learning Imitation Learning+4

Option Compatible Reward Inverse Reinforcement Learning

2019-11-07 · Rakhoon Hwang, Hanjin Lee, Hyung Ju Hwang

Reinforcement learning in complex environments is a challenging problem. In particular, the success of reinforcement learning algorithms depends on a well-designed reward function. Inverse reinforcement learning (IRL) so…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Transfer Learning

Design of APRS Modem Using IC TCM3105 and ATMega2560 Microcontroller

2019-01-10

APRS technology is still exists and still developing among amateur radio. However, in Indonesia there are still not many people who use and utilize APRS. The high price of APRS modem and difficulties of getting APRS mode…

ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrations

2019-10-26 · Daniel Seita, David Chan, Roshan Rao, Chen Tang 외

Learning from demonstrations is a popular tool for accelerating and reducing the exploration requirements of reinforcement learning. When providing expert demonstrations to human students, we know that the demonstrations…

Atari Games Deep Reinforcement Learning Q-Learning reinforcement-learning+2