Deep Variational Reinforcement Learning for POMDPs

2018-06-06 · ICML 2018 7 · Maximilian Igl, Luisa Zintgraf, Tuan Anh Le, Frank Wood, Shimon Whiteson

Many real-world sequential decision making problems are partially observable by nature, and the environment model is typically unknown. Consequently, there is great need for reinforcement learning methods that can tackle such problems given only a stream of incomplete and noisy observations. In this paper, we propose deep variational reinforcement learning (DVRL), which introduces an inductive bias that allows an agent to learn a generative model of the environment and perform inference in that model to effectively aggregate the available information. We develop an n-step approximation to the evidence lower bound (ELBO), allowing the model to be trained jointly with the policy. This ensures that the latent state representation is suitable for the control task. In experiments on Mountain Hike and flickering Atari we show that our method outperforms previous approaches relying on recurrent neural networks to encode the past.

📄 PDF Abstract BibTeX arXiv:1806.02426

Code (1)

maximilianigl/DVRL 공식 구현 pytorch

Tasks

Decision MakingInductive Biasreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sequential Decision Making

SVQN: Sequential Variational Soft Q-Learning Networks

2020-01-01 · ICLR 2020 1 · Shiyu Huang, Hang Su, Jun Zhu, Ting Chen

Partially Observable Markov Decision Processes (POMDPs) are popular and flexible models for real-world decision-making applications that demand the information from past observations to make optimal decisions. Standard r…

Decision Making Q-Learning reinforcement-learning Reinforcement Learning+1

Less Suboptimal Learning and Control in Variational POMDPs

2021-03-09 · ICLR Workshop SSL-RL 2021 5 · Baris Kayalibay, Atanas Mirchev, Patrick van der Smagt, Justin Bayer

A recently uncovered pitfall in learning generative models with amortised variational inference, the conditioning gap, questions common practices in model-based reinforcement learning. Withholding a part of the quantitie…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Finite-State Controllers for (Hidden-Model) POMDPs using Deep Reinforcement Learning

2026-02-09 · David Hudák, Maris F. L. Galesloot, Martin Tappler, Martin Kurečka 외 arxiv

Solving partially observable Markov decision processes (POMDPs) requires computing policies under imperfect state information. Despite recent advances, the scalability of existing POMDP solvers remains limited. Moreover,…

Reinforcement Learning

Variational Inference for Data-Efficient Model Learning in POMDPs

2018-05-23 · Sebastian Tschiatschek, Kai Arulkumaran, Jan Stühmer, Katja Hofmann

Partially observable Markov decision processes (POMDPs) are a powerful abstraction for tasks that require decision making under uncertainty, and capture a wide range of real world tasks. Today, effective planning approac…

Decision Making Decision Making Under Uncertainty Open-Ended Question Answering Variational Inference

Sample-Efficient Reinforcement Learning of Undercomplete POMDPs

2020-06-22 · NeurIPS 2020 12 · Chi Jin, Sham M. Kakade, Akshay Krishnamurthy, Qinghua Liu

Partial observability is a common challenge in many reinforcement learning applications, which requires an agent to maintain memory, infer latent states, and integrate this past information into exploration. This challen…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)