Reinforcement Learning State Estimation for High-Dimensional Nonlinear Systems
In high-dimensional nonlinear systems such as fluid flows, the design of state estimators such as Kalman filters relies on a reduced-order model (ROM) of the dynamics. However, ROMs are prone to large errors, which negatively affects the performance of the estimator. Here, we introduce the reinforcement learning reduced-order estimator (RL-ROE), a ROM-based estimator in which the data assimilation feedback term is given by a nonlinear stochastic policy trained through reinforcement learning. The flexibility of the nonlinear policy enables the RL-ROE to compensate for errors of the ROM, while still taking advantage of the imperfect knowledge of the dynamics. We show that the trained RL-ROE is able to outperform a Kalman filter designed using the same ROM, and displays robust estimation performance with respect to different reference trajectories and initial state estimates.
Code (0)
등록된 구현이 없습니다.
Tasks
reinforcement-learningReinforcement LearningReinforcement Learning (RL)State EstimationVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Reinforcement learning-based estimation for partial differential equations
In systems governed by nonlinear partial differential equations such as fluid flows, the design of state estimators such as Kalman filters relies on a reduced-order model (ROM) that projects the original high-dimensional…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Koopman-based Deep Learning for Nonlinear System Estimation
Nonlinear differential equations are encountered as models of fluid flow, spiking neurons, and many other systems of interest in the real world. Common features of these systems are that their behaviors are difficult to …
Deep LearningTransfer LearningReinforcement Learning with Function Approximation: From Linear to Nonlinear
Function approximation has been an indispensable component in modern reinforcement learning algorithms designed to tackle problems with large state spaces in high dimensions. This paper reviews recent results on error an…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Reinforcement learning based data assimilation for unknown state model
Data assimilation (DA) has increasingly emerged as a critical tool for state estimation across a wide range of applications. It is significantly challenging when the governing equations of the underlying dynamics are unk…
Reinforcement LearningData-driven parallel Koopman subsystem modeling and distributed moving horizon state estimation for large-scale nonlinear processes
In this work, we consider a state estimation problem for large-scale nonlinear processes in the absence of first-principles process models. By exploiting process operation data, both process modeling and state estimation…
Chemical ProcessState Estimation