Supervised Learning for Stochastic Optimal Control
Supervised machine learning is powerful. In recent years, it has enabled massive breakthroughs in computer vision and natural language processing. But leveraging these advances for optimal control has proved difficult. Data is a key limiting factor. Without access to the optimal policy, value function, or demonstrations, how can we fit a policy? In this paper, we show how to automatically generate supervised learning data for a class of continuous-time nonlinear stochastic optimal control problems. In particular, applying the Feynman-Kac theorem to a linear reparameterization of the Hamilton-Jacobi-Bellman PDE allows us to sample the value function by simulating a stochastic process. Hardware accelerators like GPUs could rapidly generate a large amount of this training data. With this data in hand, stochastic optimal control becomes supervised learning.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Online Data Poisoning Attack
We study data poisoning attacks in the online setting where training items arrive sequentially, and the attacker may perturb the current item to manipulate online learning. Importantly, the attacker has no knowledge of f…
Data PoisoningDeep Reinforcement LearningModel Predictive Controlreinforcement-learning+2Online Data Poisoning Attacks
We study data poisoning attacks in the online learning setting, where training data arrive sequentially, and the attacker is eavesdropping the data stream and has the ability to contaminate the current data point to affe…
Data PoisoningDeep Reinforcement LearningModel Predictive Controlreinforcement-learning+1On Policy Stochasticity in Mutual Information Optimal Control of Linear Systems
In recent years, mutual information optimal control has been proposed as an extension of maximum entropy optimal control. Both approaches introduce regularization terms to render the policy stochastic, and it is importan…
Self-supervised In-context Operator Learning for Stochastic Mean-Field Control
Stochastic mean-field control (MFC) provides a fundamental framework for coordinating large populations of interacting agents under uncertainty, with a wide range of applications. Existing numerical and deep-learning met…
Zero-shot GeneralizationA Backward Simulation Method for Stochastic Optimal Control Problems
A number of optimal decision problems with uncertainty can be formulated into a stochastic optimal control framework. The Least-Squares Monte Carlo (LSMC) algorithm is a popular numerical method to approach solutions of …