Model predictive control-based value estimation for efficient reinforcement learning

2023-10-25 · Qizhen Wu, Kexin Liu, Lei Chen

Reinforcement learning suffers from limitations in real practices primarily due to the number of required interactions with virtual environments. It results in a challenging problem because we are implausible to obtain a local optimal strategy with only a few attempts for many learning methods. Hereby, we design an improved reinforcement learning method based on model predictive control that models the environment through a data-driven approach. Based on the learned environment model, it performs multi-step prediction to estimate the value function and optimize the policy. The method demonstrates higher learning efficiency, faster convergent speed of strategies tending to the local optimal value, and less sample capacity space required by experience replay buffers. Experimental results, both in classic databases and in a dynamic obstacle avoidance scenario for an unmanned aerial vehicle, validate the proposed approaches.

📄 PDF Abstract BibTeX arXiv:2310.16646

Code (1)

Wu-duanduan/MPC_based-RL 공식 구현 pytorch

Tasks

Model Predictive Controlreinforcement-learning Reinforcement Learning

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…

Predictive reinforcement learning based adaptive PID controller

2025-06-10 · Chaoqun Ma, Zhiyong Zhang

Purpose: This study aims to address the challenges of controlling unstable and nonlinear systems by proposing an adaptive PID controller based on predictive reinforcement learning (PRL-PID), where the PRL-PID combines th…

reinforcement-learning Reinforcement Learning

Deep Value Model Predictive Control

2019-10-08 · Farbod Farshidian, David Hoeller, Marco Hutter

In this paper, we introduce an actor-critic algorithm called Deep Value Model Predictive Control (DMPC), which combines model-based trajectory optimization with value function estimation. The DMPC actor is a Model Predic…

model Model Predictive Control

Blending MPC & Value Function Approximation for Efficient Reinforcement Learning

2020-12-10 · ICLR 2021 1 · Mohak Bhardwaj, Sanjiban Choudhury, Byron Boots

Model-Predictive Control (MPC) is a powerful tool for controlling complex, real-world systems that uses a model to make predictions about future behavior. For each state encountered, MPC solves an online optimization pro…

Model Predictive Control reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Interpretable Stochastic Model Predictive Control using Distributional Reinforced Estimation for Quadrotor Tracking Systems

2022-05-14 · Yanran Wang, James O'Keeffe, Qiuchen Qian, David Boyle

This paper presents a novel trajectory tracker for autonomous quadrotor navigation in dynamic and complex environments. The proposed framework integrates a distributional Reinforcement Learning (RL) estimator for unknown…

Distributional Reinforcement Learning Model Predictive Control Reinforcement Learning (RL)

Collaborative Weighting with Pessimistic Critic for Mitigating Overestimation in Off-Policy Reinforcement Learning

2026-07-29 · Gong Gao, Xiao Lai, Ziqi Xie, Guojie Chen 외 arxiv

Deep off-policy reinforcement learning algorithms for continuous control typically rely on neural value function approximation to guide policy improvement. However, temporal-difference (TD) learning introduces noisy targ…

Reinforcement Learning Continuous Control