Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs
We present two elegant solutions for modeling continuous-time dynamics, in a novel model-based reinforcement learning (RL) framework for semi-Markov decision processes (SMDPs), using neural ordinary differential equations (ODEs). Our models accurately characterize continuous-time dynamics and enable us to develop high-performing policies using a small amount of data. We also develop a model-based approach for optimizing time schedules to reduce interaction rates with the environment while maintaining the near-optimal performance, which is not possible for model-free methods. We experimentally demonstrate the efficacy of our methods across various continuous-time domains.
Code (1)
Tasks
Model-based Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning
We propose a novel generalization of constrained Markov decision processes (CMDPs) that we call the \emph{semi-infinitely constrained Markov decision process} (SICMDP). Particularly, we consider a continuum of constraint…
Decision MakingDeep Reinforcement LearningModel-based Reinforcement Learningreinforcement-learning+2Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes
Off-policy evaluation (OPE) in reinforcement learning allows one to evaluate novel decision policies without needing to conduct exploration, which is often costly or otherwise infeasible. We consider for the first time t…
Off-policy evaluationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Average-reward reinforcement learning in semi-Markov decision processes via relative value iteration
This paper applies the authors' recent results on asynchronous stochastic approximation (SA) in the Borkar-Meyn framework to reinforcement learning in average-reward semi-Markov decision processes (SMDPs). We establish t…
Reinforcement LearningMarkov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processes
Our work aims at developing reinforcement learning algorithms that do not rely on the Markov assumption. We consider the class of Non-Markov Decision Processes where histories can be abstracted into a finite set of state…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Automatic Double Reinforcement Learning in Semiparametric Markov Decision Processes with Applications to Long-Term Causal Inference
Estimating long-term causal effects from short-term data is essential for decision-making in healthcare, economics, and industry, where long-term follow-up is often infeasible. Markov Decision Processes (MDPs) offer a pr…
Causal InferenceDimensionality ReductionDomain AdaptationModel Selection