Entropy-Controlled Intrinsic Motivation Reinforcement Learning for Quadruped Robot Locomotion in Complex Terrains

2025-12-06 · Wanru Gong, Xinyi Zheng, Yuan Hui, Zhongjun Li, Weiqiang Wang, Xiaoqing Zhu arxiv

Learning is the basis of both biological and artificial systems when it comes to mimicking intelligent behaviors. From the classical PPO (Proximal Policy Optimization), there is a series of deep reinforcement learning algorithms which are widely used in training locomotion policies for quadrupedal robots because of their stability and sample efficiency. However, among all these variants, experiments and simulations often converge prematurely, leading to suboptimal locomotion and reduced task performance. Therefore, in this paper, we introduce Entropy-Controlled Intrinsic Motivation (ECIM), an entropy-based reinforcement learning algorithm in contrast with the PPO series, that can reduce premature convergence by combining intrinsic motivation with adaptive exploration. For experiments, in order to parallel with other baselines, we chose to apply it in Isaac Gym across six terrain categories: upward slopes, downward slopes, uneven rough terrain, ascending stairs, descending stairs, and flat ground as widely used. For comparison, our experiments consistently achieve better performance: task rewards increase by 4--12%, peak body pitch oscillation is reduced by 23--29%, joint acceleration decreases by 20--32%, and joint torque consumption declines by 11--20%. Overall, our model ECIM, by combining entropy control and intrinsic motivation control, achieves better results in stability across different terrains for quadrupedal locomotion, and at the same time reduces energetic cost and makes it a practical choice for complex robotic control tasks.

📄 PDF Abstract BibTeX arXiv:2512.06486

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning

Constrained Intrinsic Motivation for Reinforcement Learning

2024-07-12 · Xiang Zheng, Xingjun Ma, Chao Shen, Cong Wang

This paper investigates two fundamental problems that arise when utilizing Intrinsic Motivation (IM) for reinforcement learning in Reward-Free Pre-Training (RFPT) tasks and Exploration with Intrinsic Motivation (EIM) tas…

MuJoCo reinforcement-learning Reinforcement Learning

Intrinsic motivation as constrained entropy maximization

2025-02-05 · Alex B. Kiefer

"Intrinsic motivation" refers to the capacity for intelligent systems to be motivated endogenously, i.e. by features of agential architecture itself rather than by learned associations between action and reward. This pap…

Surprise-Adaptive Intrinsic Motivation for Unsupervised Reinforcement Learning

2024-05-27 · Adriana Hugessen, Roger Creus Castanyer, Faisal Mohamed, Glen Berseth

Both entropy-minimizing and entropy-maximizing (curiosity) objectives for unsupervised reinforcement learning (RL) have been shown to be effective in different environments, depending on the environment's level of natura…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Unsupervised Reinforcement Learning

Off-Policy Maximum Entropy RL with Future State and Action Visitation Measures

2024-12-09 · Adrien Bolland, Gaspard Lambrechts, Damien Ernst

We introduce a new maximum entropy reinforcement learning framework based on the distribution of states and actions visited by a policy. More precisely, an intrinsic reward function is added to the reward function of the…

reinforcement-learning Reinforcement Learning

Intrinsically-Motivated Reinforcement Learning: A Brief Introduction

2022-03-03 · Mingqi Yuan

Reinforcement learning (RL) is one of the three basic paradigms of machine learning. It has demonstrated impressive performance in many complex tasks like Go and StarCraft, which is increasingly involved in smart manufac…

Autonomous Driving reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1