Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion
In this paper, we propose a locomotion training framework where a control policy and a state estimator are trained concurrently. The framework consists of a policy network which outputs the desired joint positions and a state estimation network which outputs estimates of the robot's states such as the base linear velocity, foot height, and contact probability. We exploit a fast simulation environment to train the networks and the trained networks are transferred to the real robot. The trained policy and state estimator are capable of traversing diverse terrains such as a hill, slippery plate, and bumpy road. We also demonstrate that the learned policy can run at up to 3.75 m/s on normal flat ground and 3.54 m/s on a slippery plate with the coefficient of friction of 0.22.
Code (1)
Tasks
FrictionState EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
State-Conditioned Adversarial Subgoal Generation
Hierarchical reinforcement learning (HRL) proposes to solve difficult tasks by performing decision-making and control at successively higher levels of temporal abstraction. However, off-policy HRL often suffers from the …
continuous-controlContinuous ControlDecision MakingHierarchical Reinforcement Learning+2Bridging Adaptivity and Safety: Learning Agile Collision-Free Locomotion Across Varied Physics
Real-world legged locomotion systems often need to reconcile agility and safety for different scenarios. Moreover, the underlying dynamics are often unknown and time-variant (e.g., payload, friction). In this paper, we i…
FrictionEstimator-Coupled Reinforcement Learning for Robust Purely Tactile In-Hand Manipulation
This paper identifies and addresses the problems with naively combining (reinforcement) learning-based controllers and state estimators for robotic in-hand manipulation. Specifically, we tackle the challenging task of pu…
GPUreinforcement-learningState EstimationLearn to Exceed: Stereo Inverse Reinforcement Learning with Concurrent Policy Optimization
In this paper, we study the problem of obtaining a control policy that can mimic and then outperform expert demonstrations in Markov decision processes where the reward function is unknown to the learning agent. One main…
reinforcement-learningReinforcement Learning (RL)Clipped Action Policy Gradient
Many continuous control tasks have bounded action spaces. When policy gradient methods are applied to such tasks, out-of-bound actions need to be clipped before execution, while policies are usually optimized as if the a…
continuous-controlContinuous ControlPolicy Gradient Methods