Variational Inference for Data-Efficient Model Learning in POMDPs

2018-05-23 · Sebastian Tschiatschek, Kai Arulkumaran, Jan Stühmer, Katja Hofmann

Partially observable Markov decision processes (POMDPs) are a powerful abstraction for tasks that require decision making under uncertainty, and capture a wide range of real world tasks. Today, effective planning approaches exist that generate effective strategies given black-box models of a POMDP task. Yet, an open question is how to acquire accurate models for complex domains. In this paper we propose DELIP, an approach to model learning for POMDPs that utilizes amortized structured variational inference. We empirically show that our model leads to effective control strategies when coupled with state-of-the-art planners. Intuitively, model-based approaches should be particularly beneficial in environments with changing reward structures, or where rewards are initially unknown. Our experiments confirm that DELIP is particularly effective in this setting.

📄 PDF Abstract BibTeX arXiv:1805.09281

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingDecision Making Under UncertaintyOpen-Ended Question AnsweringVariational Inference

SVQN: Sequential Variational Soft Q-Learning Networks

2020-01-01 · ICLR 2020 1 · Shiyu Huang, Hang Su, Jun Zhu, Ting Chen

Partially Observable Markov Decision Processes (POMDPs) are popular and flexible models for real-world decision-making applications that demand the information from past observations to make optimal decisions. Standard r…

Decision Making Q-Learning reinforcement-learning Reinforcement Learning+1

Less Suboptimal Learning and Control in Variational POMDPs

2021-03-09 · ICLR Workshop SSL-RL 2021 5 · Baris Kayalibay, Atanas Mirchev, Patrick van der Smagt, Justin Bayer

A recently uncovered pitfall in learning generative models with amortised variational inference, the conditioning gap, questions common practices in model-based reinforcement learning. Withholding a part of the quantitie…

Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Optimizing Sequential Medical Treatments with Auto-Encoding Heuristic Search in POMDPs

2019-05-17 · Luchen Li, Matthieu Komorowski, Aldo A. Faisal

Health-related data is noisy and stochastic in implying the true physiological states of patients, limiting information contained in single-moment observations for sequential clinical decision making. We model patient-cl…

Decision Making Heuristic Search

Flow-based Recurrent Belief State Learning for POMDPs

2022-05-23 · Xiaoyu Chen, Yao Mu, Ping Luo, Shengbo Li 외

Partially Observable Markov Decision Process (POMDP) provides a principled and generic framework to model real world sequential decision making processes but yet remains unsolved, especially for high dimensional continuo…

Decision Making Sequential Decision Making Variational Inference

Stick-Breaking Policy Learning in Dec-POMDPs

2015-05-01 · Miao Liu, Christopher Amato, Xuejun Liao, Lawrence Carin 외

Expectation maximization (EM) has recently been shown to be an efficient algorithm for learning finite-state controllers (FSCs) in large decentralized POMDPs (Dec-POMDPs). However, current methods use fixed-size FSCs and…