Reduced-Order Model-Guided Reinforcement Learning for Demonstration-Free Humanoid Locomotion
We introduce Reduced-Order Model-Guided Reinforcement Learning (ROM-GRL), a two-stage reinforcement learning framework for humanoid walking that requires no motion capture data or elaborate reward shaping. In the first stage, a compact 4-DOF (four-degree-of-freedom) reduced-order model (ROM) is trained via Proximal Policy Optimization. This generates energy-efficient gait templates. In the second stage, those dynamically consistent trajectories guide a full-body policy trained with Soft Actor--Critic augmented by an adversarial discriminator, ensuring the student's five-dimensional gait feature distribution matches the ROM's demonstrations. Experiments at 1 meter-per-second and 4 meter-per-second show that ROM-GRL produces stable, symmetric gaits with substantially lower tracking error than a pure-reward baseline. By distilling lightweight ROM guidance into high-dimensional policies, ROM-GRL bridges the gap between reward-only and imitation-based locomotion methods, enabling versatile, naturalistic humanoid behaviors without any human demonstrations.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
GPU-Parallel Multi-Task Reinforcement Learning with Demonstration Guided Policy Optimization
Large scale GPU-parallel reinforcement learning has changed what can be trained in robot simulation, yet most systems still optimize one specialist policy per task. We propose a construction methodology for turning struc…
Reinforcement LearningDemonstration-Guided Reinforcement Learning with Learned Skills
Demonstration-guided reinforcement learning (RL) is a promising approach for learning complex behaviors by leveraging both reward feedback and a set of target task demonstrations. Prior approaches for demonstration-guide…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Robot ManipulationDemonstration-guided Deep Reinforcement Learning for Coordinated Ramp Metering and Perimeter Control in Large Scale Networks
Effective traffic control methods have great potential in alleviating network congestion. Existing literature generally focuses on a single control approach, while few studies have explored the effectiveness of integrate…
Deep Reinforcement LearningLeveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewards
We propose a general and model-free approach for Reinforcement Learning (RL) on real robotics with sparse rewards. We build upon the Deep Deterministic Policy Gradient (DDPG) algorithm to use demonstrations. Both demonst…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Sketch-to-Skill: Bootstrapping Robot Learning with Human Drawn Trajectory Sketches
Training robotic manipulation policies traditionally requires numerous demonstrations and/or environmental rollouts. While recent Imitation Learning (IL) and Reinforcement Learning (RL) methods have reduced the number of…
Imitation Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)