Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Control
Reinforcement learning (RL) has recently proven great success in various domains. Yet, the design of the reward function requires detailed domain expertise and tedious fine-tuning to ensure that agents are able to learn the desired behaviour. Using a sparse reward conveniently mitigates these challenges. However, the sparse reward represents a challenge on its own, often resulting in unsuccessful training of the agent. In this paper, we therefore address the sparse reward problem in RL. Our goal is to find an effective alternative to reward shaping, without using costly human demonstrations, that would also be applicable to a wide range of domains. Hence, we propose to use model predictive control~(MPC) as an experience source for training RL agents in sparse reward environments. Without the need for reward shaping, we successfully apply our approach in the field of mobile robot navigation both in simulation and real-world experiments with a Kuboki Turtlebot 2. We furthermore demonstrate great improvement over pure RL algorithms in terms of success rate as well as number of collisions and timeouts. Our experiments show that MPC as an experience source improves the agent's learning process for a given task in the case of sparse rewards.
Code (0)
등록된 구현이 없습니다.
Tasks
Model Predictive Controlreinforcement-learningReinforcement LearningReinforcement Learning (RL)Robot NavigationSimilar Papers 제목 키워드 기반
Predictive Coding for Boosting Deep Reinforcement Learning with Sparse Rewards
While recent progress in deep reinforcement learning has enabled robots to learn complex behaviors, tasks with long horizons and sparse rewards remain an ongoing challenge. In this work, we propose an effective reward sh…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Residual MPC: Blending Reinforcement Learning with GPU-Parallelized Model Predictive Control
Model Predictive Control (MPC) provides interpretable, tunable locomotion controllers grounded in physical models, but its robustness depends on frequent replanning and is limited by model mismatch and real-time computat…
Reinforcement LearningActive Predicting Coding: Brain-Inspired Reinforcement Learning for Sparse Reward Robotic Control Problems
In this article, we propose a backpropagation-free approach to robotic control through the neuro-cognitive computational framework of neural generative coding (NGC), designing an agent built completely from powerful pred…
reinforcement-learningReinforcement Learning (RL)Towards Active Vision for Action Localization with Reactive Control and Predictive Learning
Visual event perception tasks such as action localization have primarily focused on supervised learning settings under a static observer, i.e., the camera is static and cannot be controlled by an algorithm. They are ofte…
Action LocalizationDiversityObject TrackingSelf-Imitation Learning for Robot Tasks with Sparse and Delayed Rewards
The application of reinforcement learning (RL) in robotic control is still limited in the environments with sparse and delayed rewards. In this paper, we propose a practical self-imitation learning method named Self-Imit…
Imitation LearningMuJoCoReinforcement Learning (RL)