Actor-Critic Algorithm for High-dimensional PDEs
We develop a machine learning model to effectively solve high-dimensional nonlinear parabolic partial differential equations (PDE). We use Feynman-Kac formula to reformulate PDE into the equivalent stochastic control problem governed by a Backward Stochastic Differential Equation (BSDE) system. Our model is designed to maximally exploit the Markovian property of the BSDE system and utilizes an Actor-Critic network architecture, which is novel in the high dimensional PDE literature. We show that our algorithm design leads to a significant speedup with higher accuracy level compared to other neural network solvers. Our model advances the state-of-the-art machine learning PDE solvers in a few aspects: 1) the trainable parameters are reduced by $N$ times, where $N$ is the number of steps to discretize the PDE in time, 2) the model convergence rate is an order of magnitude faster, 3) our model has fewer tuning hyperparameters. We demonstrate the performance improvements by solving six equations including Hamilton-Jacobian-Bellman equation, Allen-Cahn equation and Black-Scholes equation, all with dimensions on the order of 100. Those equations in high dimensions have wide applications in control theory, material science and Quantitative finance.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Actor-Critic Algorithm for High-dimensional Partial Differential Equations
We develop a deep learning model to effectively solve high-dimensional nonlinear parabolic partial differential equations (PDE). We follow Feynman-Kac formula to reformulate PDE into the equivalent stochastic control pro…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Vocal Bursts Intensity PredictionIntegration Matters for Learning PDEs with Backwards SDEs
Backward stochastic differential equation (BSDE)-based deep learning methods provide an alternative to Physics-Informed Neural Networks (PINNs) for solving high-dimensional partial differential equations (PDEs), offering…
Neural Actor-Critic Methods for Hamilton-Jacobi-Bellman PDEs: Asymptotic Analysis and Numerical Studies
We mathematically analyze and numerically study an actor-critic machine learning algorithm for solving high-dimensional Hamilton-Jacobi-Bellman (HJB) partial differential equations from stochastic control theory. The arc…
Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies
Optimal control of parametric partial differential equations (PDEs) is crucial in many applications in engineering and science. In recent years, the progress in scientific machine learning has opened up new frontiers for…
Deep Reinforcement LearningDictionary LearningDGM: A deep learning algorithm for solving partial differential equations
High-dimensional PDEs have been a longstanding computational challenge. We propose to solve high-dimensional PDEs by approximating the solution with a deep neural network which is trained to satisfy the differential oper…
Deep Learning