paper-with-me

Papers

An Efficient On-Policy Deep Learning Framework for Stochastic Optimal Control

2024-10-07 · Mengjian Hua, Mathieu Laurière, Eric Vanden-Eijnden

We present a novel on-policy algorithm for solving stochastic optimal control (SOC) problems. By leveraging the Girsanov theorem, our method directly computes on-policy gradients of the SOC objective without expensive backpropagation through stochastic differential equations or adjoint problem solutions. This approach significantly accelerates the optimization of neural network control policies while scaling efficiently to high-dimensional problems and long time horizons. We evaluate our method on classical SOC benchmarks as well as applications to sampling from unnormalized distributions via Schr\"odinger-F\"ollmer processes and fine-tuning pre-trained diffusion models. Experimental results demonstrate substantial improvements in both computational speed and memory efficiency compared to existing approaches.

📄 PDF Abstract BibTeX arXiv:2410.05163

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Learning

Methods 이 논문이 사용한 방법론

SPEED The monocular depth estimation (MDE) is the task of estimating depth from a single frame. This information is an essential knowledge in many computer vision tasks such as scene…
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee

2023-02-11 · Mo Zhou, Jianfeng Lu

We consider policy gradient methods for stochastic optimal control problem in continuous time. In particular, we analyze the gradient flow for the control, viewed as a continuous time limit of the policy gradient method.…

Policy Gradient Methods

Continuous Policy and Value Iteration for Stochastic Control Problems and Its Convergence

2025-06-09 · Qi Feng, Gu Wang

We introduce a continuous policy-value iteration algorithm where the approximations of the value function of a stochastic control problem and the optimal control are simultaneously updated through Langevin-type dynamics.…

On Policy Stochasticity in Mutual Information Optimal Control of Linear Systems

2025-07-29 · Shoju Enami, Kenji Kashima arxiv

In recent years, mutual information optimal control has been proposed as an extension of maximum entropy optimal control. Both approaches introduce regularization terms to render the policy stochastic, and it is importan…

Variational Policy for Guiding Point Processes

2017-01-30 · ICML 2017 8 · Yichen Wang, Grady Williams, Evangelos Theodorou, Le Song

Temporal point processes have been widely applied to model event sequence data generated by online users. In this paper, we consider the problem of how to design the optimal control policy for point processes, such that …

Point ProcessesVariational Inference

Inferring the Optimal Policy using Markov Chain Monte Carlo

2019-11-16 · Brandon Trabucco, Albert Qu, Simon Li, Ganeshkumar Ashokavardhanan

This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement le…

Reinforcement Learning