Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning

2017-05-01 · Thomas M. Moerland, Joost Broekens, Catholijn M. Jonker

In this paper we study how to learn stochastic, multimodal transition dynamics in reinforcement learning (RL) tasks. We focus on evaluating transition function estimation, while we defer planning over this model to future work. Stochasticity is a fundamental property of many task environments. However, discriminative function approximators have difficulty estimating multimodal stochasticity. In contrast, deep generative models do capture complex high-dimensional outcome distributions. First we discuss why, amongst such models, conditional variational inference (VI) is theoretically most appealing for model-based RL. Subsequently, we compare different VI models on their ability to learn complex stochasticity on simulated functions, as well as on a typical RL gridworld with multimodal dynamics. Results show VI successfully predicts multimodal outcomes, but also robustly ignores these for deterministic parts of the transition dynamics. In summary, we show a robust method to learn multimodal transitions using function approximation, which is a key preliminary for model-based RL in stochastic domains.

📄 PDF Abstract BibTeX arXiv:1705.00470

Code (1)

tmoer/multimodal_varinf 공식 구현 tf

Tasks

Model-based Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Variational Inference

Quantifying Multimodality in World Models

2021-12-14 · Andreas Sedlmeier, Michael Kölle, Robert Müller, Leo Baudrexel 외

Model-based Deep Reinforcement Learning (RL) assumes the availability of a model of an environment's underlying transition dynamics. This model can be used to predict future effects of an agent's possible actions. When n…

Deep Reinforcement Learning Reinforcement Learning (RL)

Variational Dynamic for Self-Supervised Exploration in Deep Reinforcement Learning

2020-10-17 · Chenjia Bai, Peng Liu, Kaiyu Liu, Lingxiao Wang 외

Efficient exploration remains a challenging problem in reinforcement learning, especially for tasks where extrinsic rewards from environments are sparse or even totally disregarded. Significant advances based on intrinsi…

Deep Reinforcement Learning Efficient Exploration reinforcement-learning Reinforcement Learning+2

Reward-Consistent Dynamics Models are Strongly Generalizable for Offline Reinforcement Learning

2023-10-09 · Fan-Ming Luo, Tian Xu, Xingchen Cao, Yang Yu

Learning a precise dynamics model can be crucial for offline reinforcement learning, which, unfortunately, has been found to be quite challenging. Dynamics models that are learned by fitting historical transitions often …

D4RL Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning

Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch

2020-07-02 · NeurIPS 2021 12 · Luca Viano, Yu-Ting Huang, Parameswaran Kamalaruban, Adrian Weller 외

We study the inverse reinforcement learning (IRL) problem under a transition dynamics mismatch between the expert and the learner. Specifically, we consider the Maximum Causal Entropy (MCE) IRL learner model and provide …

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Bridging Dynamics Gaps via Diffusion Schrödinger Bridge for Cross-Domain Reinforcement Learning

2026-02-27 · Hanping Zhang, Yuhong Guo arxiv

Cross-domain reinforcement learning (RL) aims to learn transferable policies under dynamics shifts between source and target domains. A key challenge lies in the lack of target-domain environment interaction and reward s…

Reinforcement Learning