Learning Factored Markov Decision Processes with Unawareness

2019-02-27 · Craig Innes, Alex Lascarides

Methods for learning and planning in sequential decision problems often assume the learner is aware of all possible states and actions in advance. This assumption is sometimes untenable. In this paper, we give a method to learn factored markov decision problems from both domain exploration and expert assistance, which guarantees convergence to near-optimal behaviour, even when the agent begins unaware of factors critical to success. Our experiments show our agent learns optimal behaviour on small and large problems, and that conserving information on discovering new possibilities results in faster convergence.

📄 PDF Abstract BibTeX arXiv:1902.10619

Code (0)

등록된 구현이 없습니다.

MDPs with Unawareness

2014-07-27 · Joseph Y. Halpern, Nan Rong, Ashutosh Saxena

Markov decision processes (MDPs) are widely used for modeling decision-making problems in robotics, automated control, and economics. Traditional MDPs assume that the decision maker (DM) knows all states and actions. How…

Decision Making

Towards Minimax Optimal Reinforcement Learning in Factored Markov Decision Processes

2020-06-24 · NeurIPS 2020 12 · Yi Tian, Jian Qian, Suvrit Sra

We study minimax optimal reinforcement learning in episodic factored Markov decision processes (FMDPs), which are MDPs with conditionally independent transition components. Assuming the factorization is known, we propose…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Optimal Sensor and Actuator Selection for Factored Markov Decision Processes: Complexity, Approximability and Algorithms

2024-07-10 · Jayanth Bhargav, Mahsa Ghasemi, Shreyas Sundaram

Factored Markov Decision Processes (fMDPs) are a class of Markov Decision Processes (MDPs) in which the states (and actions) can be factored into a set of state (and action) variables and can be encoded compactly using a…

Reinforcement Learning in Factored MDPs: Oracle-Efficient Algorithms and Tighter Regret Bounds for the Non-Episodic Setting

2020-02-06 · NeurIPS 2020 12 · Ziping Xu, Ambuj Tewari

We study reinforcement learning in non-episodic factored Markov decision processes (FMDPs). We propose two near-optimal and oracle-efficient algorithms for FMDPs. Assuming oracle access to an FMDP planner, they enjoy a B…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Polynomial Time Reinforcement Learning in Factored State MDPs with Linear Value Functions

2021-07-12 · Zihao Deng, Siddartha Devic, Brendan Juba

Many reinforcement learning (RL) environments in practice feature enormous state spaces that may be described compactly by a "factored" structure, that may be modeled by Factored Markov Decision Processes (FMDPs). We pre…

reinforcement-learning Reinforcement Learning (RL)