Theoretical and Numerical Analysis of Approximate Dynamic Programming with Approximation Errors

2014-12-18 · Ali Heydari

This study is aimed at answering the famous question of how the approximation errors at each iteration of Approximate Dynamic Programming (ADP) affect the quality of the final results considering the fact that errors at each iteration affect the next iteration. To this goal, convergence of Value Iteration scheme of ADP for deterministic nonlinear optimal control problems with undiscounted cost functions is investigated while considering the errors existing in approximating respective functions. The boundedness of the results around the optimal solution is obtained based on quantities which are known in a general optimal control problem and assumptions which are verifiable. Moreover, since the presence of the approximation errors leads to the deviation of the results from optimality, sufficient conditions for stability of the system operated by the result obtained after a finite number of value iterations, along with an estimation of its region of attraction, are derived in terms of a calculable upper bound of the control approximation error. Finally, the process of implementation of the method on an orbital maneuver problem is investigated through which the assumptions made in the theoretical developments are verified and the sufficient conditions are applied for guaranteeing stability and near optimality.

📄 PDF Abstract BibTeX arXiv:1412.6095

Code (0)

등록된 구현이 없습니다.

Unifying Value Iteration, Advantage Learning, and Dynamic Policy Programming

2017-10-30 · Tadashi Kozuno, Eiji Uchibe, Kenji Doya

Approximate dynamic programming algorithms, such as approximate value iteration, have been successfully applied to many complex reinforcement learning tasks, and a better approximate dynamic programming algorithm is expe…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep neural networks algorithms for stochastic control problems on finite horizon: convergence analysis

2018-12-11 · Côme Huré, Huyên Pham, Achref Bachouch, Nicolas Langrené

This paper develops algorithms for high-dimensional stochastic control problems based on deep learning and dynamic programming. Unlike classical approximate dynamic programming approaches, we first approximate the optima…

Deep Reinforcement Learning Quantization Reinforcement Learning

Off-Policy Temporal Difference Learning for Perturbed Markov Decision Processes: Theoretical Insights and Extensive Simulations

2025-02-25 · Ali Forootani, Raffaele Iervolino, Massimo Tipaldi, Mohammad Khosravi

Dynamic Programming suffers from the curse of dimensionality due to large state and action spaces, a challenge further compounded by uncertainties in the environment. To mitigate these issue, we explore an off-policy bas…

Partially Observable Reference Policy Programming: Solving POMDPs Sans Numerical Optimisation

2025-07-16 · Edward Kim, Hanna Kurniawati arxiv

This paper proposes Partially Observable Reference Policy Programming, a novel anytime online approximate POMDP solver which samples meaningful future histories very deeply while simultaneously forcing a gradual policy u…

Optimal Control Theoretic Neural Optimizer: From Backpropagation to Dynamic Programming

2025-10-15 · Guan-Horng Liu, Tianrong Chen, Evangelos A. Theodorou arxiv

Optimization of deep neural networks (DNNs) has been a driving force in the advancement of modern machine learning and artificial intelligence. With DNNs characterized by a prolonged sequence of nonlinear propagation, de…