Explainable Deterministic MDPs
We present a method for a certain class of Markov Decision Processes (MDPs) that can relate the optimal policy back to one or more reward sources in the environment. For a given initial state, without fully computing the value function, q-value function, or the optimal policy the algorithm can determine which rewards will and will not be collected, whether a given reward will be collected only once or continuously, and which local maximum within the value function the initial state will ultimately lead to. We demonstrate that the method can be used to map the state space to identify regions that are dominated by one reward source and can fully analyze the state space to explain all actions. We provide a mathematical framework to show how all of this is possible without first computing the optimal policy or value function.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Scalable Solution Methods for Dec-POMDPs with Deterministic Dynamics
Many high-level multi-agent planning problems, including multi-robot navigation and path planning, can be effectively modeled using deterministic actions and observations. In this work, we focus on such domains and intro…
Robot NavigationComputing the Reachability Value of Posterior-Deterministic POMDPs
Partially observable Markov decision processes (POMDPs) are a fundamental model for sequential decision-making under uncertainty. However, many verification and synthesis problems for POMDPs are undecidable or intractabl…
A Finite-State Controller Based Offline Solver for Deterministic POMDPs
Deterministic partially observable Markov decision processes (DetPOMDPs) often arise in planning problems where the agent is uncertain about its environmental state but can act and observe deterministically. In this pape…
Regret Analysis in Deterministic Reinforcement Learning
We consider Markov Decision Processes (MDPs) with deterministic transitions and study the problem of regret minimization, which is central to the analysis and design of optimal learning algorithms. We present logarithmic…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Counterfactual equivalence for POMDPs, and underlying deterministic environments
Partially Observable Markov Decision Processes (POMDPs) are rich environments often used in machine learning. But the issue of information and causal structures in POMDPs has been relatively little studied. This paper pr…
BIG-bench Machine Learningcounterfactual