Learning From Failures: Efficient Reinforcement Learning Control with Episodic Memory
Reinforcement learning has achieved remarkable success in robot learning. However, under challenging exploration and contact-rich dynamics, early-stage training is frequently dominated by premature terminations such as collisions and falls. As a result, learning is overwhelmed by short-horizon, low-return trajectories, which hinder convergence and limit long-horizon exploration. To alleviate this issue, we propose a technique called Failure Episodic Memory Alert (FEMA). FEMA explicitly stores short-horizon failure experiences through an episodic memory module. During interactions, it retrieves similar failure experiences and prevents the robot from recurrently relapsing into unstable states, guiding the policy toward long-horizon trajectories with greater long-term value. FEMA can be combined easily with model-free reinforcement learning algorithms, and yields a substantial sample-efficiency improvement of 33.11% on MuJoCo tasks across several classical RL algorithms. Furthermore, integrating FEMA into a parallelized PPO training pipeline demonstrates its effectiveness on a real-world bipedal robot task.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
Model-Based Episodic Memory Induces Dynamic Hybrid Controls
Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory. We propose a new model-based episodic memory of trajectories addressing current limitations of e…
modelreinforcement-learningReinforcement LearningReinforcement Learning (RL)Solving Continuous Control with Episodic Memory
Episodic memory lets reinforcement learning algorithms remember and exploit promising experience from the past to improve agent performance. Previous works on memory mechanisms show benefits of using episodic-based data …
continuous-controlContinuous ControlOpenAI GymReinforcement Learning (RL)Efficient Replay Memory Architectures in Multi-Agent Reinforcement Learning for Traffic Congestion Control
Episodic control, inspired by the role of episodic memory in the human brain, has been shown to improve the sample inefficiency of model-free reinforcement learning by reusing high-return past experiences. However, the m…
Decision MakingMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning+1Sequential memory improves sample and memory efficiency in Episodic Control
State of the art deep reinforcement learning algorithms are sample inefficient due to the large number of episodes they require to achieve asymptotic performance. Episodic Reinforcement Learning (ERL) algorithms, inspire…
Deep Reinforcement LearningHippocampusreinforcement-learningReinforcement Learning+1Continuous Episodic Control
Non-parametric episodic memory can be used to quickly latch onto high-rewarded experience in reinforcement learning tasks. In contrast to parametric deep reinforcement learning approaches in which reward signals need to …
continuous-controlContinuous ControlDecision MakingDeep Reinforcement Learning+4