Reinforcement Learning based on MPC/MHE for Unmodeled and Partially Observable Dynamics
This paper proposes an observer-based framework for solving Partially Observable Markov Decision Processes (POMDPs) when an accurate model is not available. We first propose to use a Moving Horizon Estimation-Model Predictive Control (MHE-MPC) scheme in order to provide a policy for the POMDP problem, where the full state of the real process is not measured and necessarily known. We propose to parameterize both MPC and MHE formulations, where certain adjustable parameters are regarded for tuning the policy. In this paper, for the sake of tackling the unmodeled and partially observable dynamics, we leverage the Reinforcement Learning (RL) to tune the parameters of MPC and MHE schemes jointly, with the closed-loop performance of the policy as a goal rather than model fitting or the MHE performance. Illustrations show that the proposed approach can effectively increase the performance of close-loop control of systems formulated as POMDPs.
Code (0)
등록된 구현이 없습니다.
Tasks
Model Predictive Controlreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Modeling the effects of environmental and perceptual uncertainty using deterministic reinforcement learning dynamics with partial observability
Assessing the systemic effects of uncertainty that arises from agents' partial observation of the true states of the world is critical for understanding a wide range of scenarios. Yet, previous modeling work on agent lea…
Decision MakingSequence-level Intrinsic Exploration Model for Partially Observable Domains
Training reinforcement learning policies in partially observable domains with sparse reward signal is an important and open problem for the research community. In this paper, we introduce a new sequence-level intrinsic n…
Predictionreinforcement-learningReinforcement Learning (RL)ODE-based Recurrent Model-free Reinforcement Learning for POMDPs
Neural ordinary differential equations (ODEs) are widely recognized as the standard for modeling physical mechanisms, which help to perform approximate inference in unknown physical or biological environments. In partial…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+2End-to-End Learning of Hybrid Inverse Dynamics Models for Precise and Compliant Impedance Control
It is well-known that inverse dynamics models can improve tracking performance in robot control. These models need to precisely capture the robot dynamics, which consist of well-understood components, e.g., rigid body dy…
FrictionPropagation Networks for Model-Based Control Under Partial Observation
There has been an increasing interest in learning dynamics simulators for model-based control. Compared with off-the-shelf physics engines, a learnable simulator can quickly adapt to unseen objects, scenes, and tasks. Ho…
Deep Reinforcement LearningReinforcement Learning