A Pontryagin Perspective on Reinforcement Learning
Reinforcement learning has traditionally focused on learning state-dependent policies to solve optimal control problems in a closed-loop fashion. In this work, we introduce the paradigm of open-loop reinforcement learning where a fixed action sequence is learned instead. We present three new algorithms: one robust model-based method and two sample-efficient model-free methods. Rather than basing our algorithms on Bellman's equation from dynamic programming, our work builds on Pontryagin's principle from the theory of open-loop optimal control. We provide convergence guarantees and evaluate all methods empirically on a pendulum swing-up task, as well as on two high-dimensional MuJoCo tasks, significantly outperforming existing baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
MuJoCoreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Beyond the Bellman Recursion: A Pontryagin-Guided Framework for Non-Exponential Discounting
Most value-based and actor--critic reinforcement learning methods rely on Bellman-style recursions, yet these recursions collapse under non-exponential discounting common in human preferences and survival processes. We s…
Reinforcement LearningProbabilistic Pontryagin's Maximum Principle for Continuous-Time Model-Based Reinforcement Learning
Without exact knowledge of the true system dynamics, optimal control of non-linear continuous-time systems requires careful treatment of epistemic uncertainty. In this work, we propose a probabilistic extension to Pontry…
Model-based Reinforcement Learningreinforcement-learningReinforcement LearningPontryagin Differentiable Programming: An End-to-End Learning and Control Framework
This paper develops a Pontryagin Differentiable Programming (PDP) methodology, which establishes a unified framework to solve a broad class of learning and control tasks. The PDP distinguishes from existing methods by tw…
Reinforcement Learning for Molecular Dynamics Optimization: A Stochastic Pontryagin Maximum Principle Approach
In this paper, we present a novel reinforcement learning framework designed to optimize molecular dynamics by focusing on the entire trajectory rather than just the final molecular configuration. Leveraging a stochastic …
Decision MakingDrug DiscoveryProtein FoldingProtein Structure Prediction+3A Pontryagin Method of Model-based Reinforcement Learning via Hamiltonian Actor-Critic
Model-based reinforcement learning (MBRL) improves sample efficiency by leveraging learned dynamics models for policy optimization. However, the effectiveness of methods such as actor-critic is often limited by compoundi…
Reinforcement LearningContinuous ControlOffline RL