Continuous-Time Model-Based Reinforcement Learning

2021-02-09 · Çağatay Yıldız, Markus Heinonen, Harri Lähdesmäki

Model-based reinforcement learning (MBRL) approaches rely on discrete-time state transition models whereas physical systems and the vast majority of control tasks operate in continuous-time. To avoid time-discretization approximation of the underlying process, we propose a continuous-time MBRL framework based on a novel actor-critic method. Our approach also infers the unknown state evolution differentials with Bayesian neural ordinary differential equations (ODE) to account for epistemic uncertainty. We implement and test our method on a new ODE-RL suite that explicitly solves continuous-time control systems. Our experiments illustrate that the model is robust against irregular and noisy data, is sample-efficient, and can solve control problems which pose challenges to discrete-time MBRL methods.

📄 PDF Abstract BibTeX arXiv:2102.04764

Code (1)

cagatayyildiz/oderl 공식 구현 pytorch

Tasks

modelModel-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Continuous Time Continuous Space Homeostatic Reinforcement Learning (CTCS-HRRL) : Towards Biological Self-Autonomous Agent

2024-01-17 · Hugo Laurencon, Yesoda Bhargava, Riddhi Zantye, Charbel-Raphaël Ségerie 외

Homeostasis is a biological process by which living beings maintain their internal balance. Previous research suggests that homeostasis is a learned behaviour. Recently introduced Homeostatic Regulated Reinforcement Lear…

Decision Making reinforcement-learning Reinforcement Learning

Deterministic Policy Gradient for Reinforcement Learning with Continuous Time and State

2025-09-28 · Ziheng Cheng, Xin Guo, Yufei Zhang arxiv

The theory of continuous-time reinforcement learning (RL) has progressed rapidly in recent years. While the ultimate objective of RL is typically to learn deterministic control policies, most existing continuous-time RL …

General Reinforcement Learning

Efficient Exploration in Continuous-time Model-based Reinforcement Learning

2023-10-30 · NeurIPS 2023 11

Reinforcement learning algorithms typically consider discrete-time dynamics, even though the underlying systems are often continuous in time. In this paper, we introduce a model-based reinforcement learning algorithm tha…

Efficient Exploration Gaussian Processes Model-based Reinforcement Learning reinforcement-learning+1

Discrete-Time Mean-Variance Strategy Based on Reinforcement Learning

2023-12-24 · Xiangyu Cui, Xun Li, Yun Shi, Si Zhao

This paper studies a discrete-time mean-variance model based on reinforcement learning. Compared with its continuous-time counterpart in \cite{zhou2020mv}, the discrete-time model makes more general assumptions about the…

reinforcement-learning Reinforcement Learning

Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learning

2024-09-12 · Hanyang Zhao, Haoxian Chen, Ji Zhang, David D. Yao 외

Reinforcement Learning from human feedback (RLHF) has been shown a promising direction for aligning generative models with human intent and has also been explored in recent works for alignment of diffusion generative mod…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)