Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning
In this work, we present and study a training set-up that achieves fast policy generation for real-world robotic tasks by using massive parallelism on a single workstation GPU. We analyze and discuss the impact of different training algorithm components in the massively parallel regime on the final policy performance and training times. In addition, we present a novel game-inspired curriculum that is well suited for training with thousands of simulated robots in parallel. We evaluate the approach by training the quadrupedal robot ANYmal to walk on challenging terrain. The parallel approach allows training policies for flat terrain in under four minutes, and in twenty minutes for uneven terrain. This represents a speedup of multiple orders of magnitude compared to previous work. Finally, we transfer the policies to the real robot to validate the approach. We open-source our training code to help accelerate further research in the field of learned legged locomotion.
Code (5)
Tasks
Deep Reinforcement LearningGPUreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Practical Massively Parallel Monte-Carlo Tree Search Applied to Molecular Design
It is common practice to use large computational resources to train neural networks, as is known from many examples, such as reinforcement learning applications. However, while massively parallel computing is often used …
Combinatorial OptimizationCPULearning Sim-to-Real Humanoid Locomotion in 15 Minutes
Massively parallel simulation has reduced reinforcement learning (RL) training time for robots from days to minutes. However, achieving fast and reliable sim-to-real RL for humanoid control remains difficult due to the c…
Reinforcement LearningA Walk in the Park: Learning to Walk in 20 Minutes With Model-Free Reinforcement Learning
Deep reinforcement learning is a promising approach to learning policies in uncontrolled environments that do not require domain knowledge. Unfortunately, due to sample inefficiency, deep RL applications have primarily f…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)MO-Playground: Massively Parallelized Multi-Objective Reinforcement Learning for Robotics
Multi-objective reinforcement learning (MORL) is a powerful tool to learn Pareto-optimal policy families across conflicting objectives. However, unlike traditional RL algorithms, existing MORL algorithms do not effective…
Reinforcement LearningKamino: GPU-based Massively Parallel Simulation of Multi-Body Systems with Challenging Topologies
We present Kamino, a GPU-based physics solver for massively parallel simulations of heterogeneous highly-coupled mechanical systems. Implemented in Python using NVIDIA Warp and integrated into the Newton framework, it en…
Reinforcement Learning