Reachable Space Characterization of Markov Decision Processes with Time Variability
We propose a solution to a time-varying variant of Markov Decision Processes which can be used to address decision-theoretic planning problems for autonomous systems operating in unstructured outdoor environments. We explore the time variability property of the planning stochasticity and investigate the state reachability, based on which we then develop an efficient iterative method that offers a good trade-off between solution optimality and time complexity. The reachability space is constructed by analyzing the means and variances of states' reaching time in the future. We validate our algorithm through extensive simulations using ocean data, and the results show that our method achieves a great performance in terms of both solution quality and computing time.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Markov Abstractions for PAC Reinforcement Learning in Non-Markov Decision Processes
Our work aims at developing reinforcement learning algorithms that do not rely on the Markov assumption. We consider the class of Non-Markov Decision Processes where histories can be abstracted into a finite set of state…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)A matrix theoretic characterization of the strongly reachable subspace
In this paper, we provide novel characterizations of the weakly unobservable and the strongly reachable subspaces corresponding to a given state-space system. These characterizations provide closed-form representations f…
Perception-Aware Point-Based Value Iteration for Partially Observable Markov Decision Processes
Partially observable Markov decision processes (POMDPs) are a widely-used framework to model decision-making with uncertainty about the environment and under stochastic outcome. In conventional POMDP models, the observat…
Decision MakingFaster saddle-point optimization for solving large-scale Markov decision processes
We consider the problem of computing optimal policies in average-reward Markov decision processes. This classical problem can be formulated as a linear program directly amenable to saddle-point optimization methods, albe…
The Value Function Semi-Algebraic Set in Partially Observable Markov Decision Processes
We study the geometry of feasible value functions in infinite-horizon partially observable Markov decision processes (POMDPs) under memoryless stochastic policies. Our main contribution is a characterization of the feasi…