Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis

2018-12-11 · Côme Huré, Huyên Pham, Achref Bachouch, Nicolas Langrené

This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optimal policy by means of neural networks in the spirit of deep reinforcement learning, and then the value function by Monte Carlo regression. This is achieved in the dynamic programming recursion by performance or hybrid iteration, and regress now methods from numerical probabilities. We provide a theoretical justification of these algorithms. Consistency and rate of convergence for the control and value function estimates are analyzed and expressed in terms of the universal approximation error of the neural networks, and of the statistical error when estimating network function, leaving aside the optimization error. Numerical results on various applications are presented in a companion paper (arxiv.org/abs/1812.05916) and illustrate the performance of the proposed algorithms.

📄 PDF Abstract BibTeX arXiv:1812.04300

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Reinforcement LearningQuantization Reinforcement Learning

A Machine Learning Algorithm for Finite-Horizon Stochastic Control Problems in Economics

2024-11-13 · Xianhua Peng, Steven Kou, Lekang Zhang

We propose a machine learning algorithm for solving finite-horizon stochastic control problems based on a deep neural network representation of the optimal policy functions. The algorithm has three features: (1) It can s…

Linear convergence of a policy gradient method for some finite horizon continuous time control problems

2022-03-22 · Christoph Reisinger, Wolfgang Stockinger, Yufei Zhang

Despite its popularity in the reinforcement learning community, a provably convergent policy gradient method for continuous space-time control problems with nonlinear state dynamics has been elusive. This paper proposes …

Policy Gradient Methods reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep neural networks algorithms for stochastic control problems on finite horizon: numerical applications

2018-12-13 · Achref Bachouch, Côme Huré, Nicolas Langrené, Huyen Pham

This paper presents several numerical applications of deep learning-based algorithms that have been introduced in [HPBL18]. Numerical and comparative tests using TensorFlow illustrate the performance of our different alg…

Management Quantization

Convergence Analysis of Machine Learning Algorithms for the Numerical Solution of Mean Field Control and Games: II -- The Finite Horizon Case

2019-08-05 · René Carmona, Mathieu Laurière

We propose two numerical methods for the optimal control of McKean-Vlasov dynamics in finite time horizon. Both methods are based on the introduction of a suitable loss function defined over the parameters of a neural ne…

Periodic Finite State Controllers for Efficient POMDP and DEC-POMDP Planning

2011-12-01 · NeurIPS 2011 12 · Joni K. Pajarinen, Jaakko Peltonen

Applications such as robot control and wireless communication require planning under uncertainty. Partially observable Markov decision processes (POMDPs) plan policies for single agents under uncertainty and their decent…