Learning Semantics-Aware Locomotion Skills from Human Demonstration

2022-06-27 · Yuxiang Yang, Xiangyun Meng, Wenhao Yu, Tingnan Zhang, Jie Tan, Byron Boots

The semantics of the environment, such as the terrain type and property, reveals important information for legged robots to adjust their behaviors. In this work, we present a framework that learns semantics-aware locomotion skills from perception for quadrupedal robots, such that the robot can traverse through complex offroad terrains with appropriate speeds and gaits using perception information. Due to the lack of high-fidelity outdoor simulation, our framework needs to be trained directly in the real world, which brings unique challenges in data efficiency and safety. To ensure sample efficiency, we pre-train the perception model with an off-road driving dataset. To avoid the risks of real-world policy exploration, we leverage human demonstration to train a speed policy that selects a desired forward speed from camera image. For maximum traversability, we pair the speed policy with a gait selector, which selects a robust locomotion gait for each forward speed. Using only 40 minutes of human demonstration data, our framework learns to adjust the speed and gait of the robot based on perceived terrain semantics, and enables the robot to walk over 6km without failure at close-to-optimal speed.

📄 PDF Abstract BibTeX arXiv:2206.13631

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Learning Multi-Skill Legged Locomotion Using Conditional Adversarial Motion Priors

2025-09-26 · Ning Huang, Zhentao Xie, Qinchuan Li arxiv

Despite growing interest in developing legged robots that emulate biological locomotion for agile navigation of complex environments, acquiring a diverse repertoire of skills remains a fundamental challenge in robotics. …

Reinforcement Learning for Robotic Manipulation using Simulated Locomotion Demonstrations

2019-10-16 · Ozsel Kilinc, Giovanni Montana

Mastering robotic manipulation skills through reinforcement learning (RL) typically requires the design of shaped reward functions. Recent developments in this area have demonstrated that using sparse rewards, i.e. rewar…

Object reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Learning Whole-Body Humanoid Locomotion via Motion Generation and Motion Tracking

2026-04-19 · Zewei Zhang, Kehan Wen, Michael Xu, Junzhe He 외 arxiv

Whole-body humanoid locomotion is challenging due to high-dimensional control, morphological instability, and the need for real-time adaptation to various terrains using onboard perception. Directly applying reinforcemen…

Reinforcement Learning

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control

2024-01-30 · Zhongyu Li, Xue Bin Peng, Pieter Abbeel, Sergey Levine 외

This paper presents a comprehensive study on using deep reinforcement learning (RL) to create dynamic locomotion controllers for bipedal robots. Going beyond focusing on a single locomotion skill, we develop a general co…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)

LocoVLM: Grounding Vision and Language for Adapting Versatile Legged Locomotion Policies

2026-02-11 · I Made Aswin Nahrendra, Seunghyun Lee, Dongkyu Lee, Hyun Myung arxiv

Recent advances in legged locomotion learning are still dominated by the utilization of geometric representations of the environment, limiting the robot's capability to respond to higher-level semantics such as human ins…