Incorporating Recurrent Reinforcement Learning into Model Predictive Control for Adaptive Control in Autonomous Driving

2023-01-30 · Yuan Zhang, Joschka Boedecker, Chuxuan Li, Guyue Zhou

Model Predictive Control (MPC) is attracting tremendous attention in the autonomous driving task as a powerful control technique. The success of an MPC controller strongly depends on an accurate internal dynamics model. However, the static parameters, usually learned by system identification, often fail to adapt to both internal and external perturbations in real-world scenarios. In this paper, we firstly (1) reformulate the problem as a Partially Observed Markov Decision Process (POMDP) that absorbs the uncertainties into observations and maintains Markov property into hidden states; and (2) learn a recurrent policy continually adapting the parameters of the dynamics model via Recurrent Reinforcement Learning (RRL) for optimal and adaptive control; and (3) finally evaluate the proposed algorithm (referred as $\textit{MPC-RRL}$) in CARLA simulator and leading to robust behaviours under a wide range of perturbations.

📄 PDF Abstract BibTeX arXiv:2301.13313

Code (0)

등록된 구현이 없습니다.

Tasks

Autonomous DrivingModel Predictive Control

Methods 이 논문이 사용한 방법론

Entropy Regularization 설명 없음

PPO Proximal Policy Optimization, or PPO, is a policy gradient method for reinforcement learning. The motivation was to have an algorithm with the data efficiency and reliable…

fail 설명 없음

CARLA CARLA is an open-source simulator for autonomous driving research. CARLA has been developed from the ground up to support development, training, and validation of autonomous urban…

Predictive reinforcement learning based adaptive PID controller

2025-06-10 · Chaoqun Ma, Zhiyong Zhang

Purpose: This study aims to address the challenges of controlling unstable and nonlinear systems by proposing an adaptive PID controller based on predictive reinforcement learning (PRL-PID), where the PRL-PID combines th…

reinforcement-learning Reinforcement Learning

Recurrent Predictive State Policy Networks

2018-03-05 · ICML 2018 7 · Ahmed Hefny, Zita Marinho, Wen Sun, Siddhartha Srinivasa 외

We introduce Recurrent Predictive State Policy (RPSP) networks, a recurrent architecture that brings insights from predictive state representations to reinforcement learning in partially observable environments. Predicti…

OpenAI Gym Reinforcement Learning

P4O: Efficient Deep Reinforcement Learning with Predictive Processing Proximal Policy Optimization

2021-09-29 · Burcu Küçükoğlu, Walraaf Borkent, Bodo Rueckauer, Nasir Ahmad 외

Advances in reinforcement learning (RL) often rely on massive compute resources and remain notoriously sample inefficient. In contrast, the human brain is able to efficiently learn effective control strategies using limi…

Atari Games Deep Reinforcement Learning GPU reinforcement-learning+2

Efficient Deep Reinforcement Learning with Predictive Processing Proximal Policy Optimization

2022-11-11 · Burcu Küçükoğlu, Walraaf Borkent, Bodo Rueckauer, Nasir Ahmad 외

Atari Games Deep Reinforcement Learning GPU reinforcement-learning+2

Predictive-State Decoders: Encoding the Future into Recurrent Networks

2017-09-25 · NeurIPS 2017 12 · Arun Venkatraman, Nicholas Rhinehart, Wen Sun, Lerrel Pinto 외

Recurrent neural networks (RNNs) are a vital modeling technique that rely on internal states learned indirectly by optimization of a supervised, unsupervised, or reinforcement training loss. RNNs are used to model dynami…

Imitation Learning Reinforcement Learning