Planning in entropy-regularized Markov decision processes and games
We propose SmoothCruiser, a new planning algorithm for estimating the value function in entropy-regularized Markov decision processes and two-player games, given a generative model of the SmoothCruiser. SmoothCruiser makes use of the smoothness of the Bellman operator promoted by the regularization to achieve problem-independent sample complexity of order $\tilde{\mathcal{O}}(1/\epsilon^4)$ for a desired accuracy $\epsilon$, whereas for non-regularized settings there are no known algorithms with guaranteed polynomial sample complexity in the worst case.
Code (1)
Similar Papers 제목 키워드 기반
Planning in entropy-regularized Markov decision processes and games
We propose SmoothCruiser, a new planning algorithm for estimating the value function in entropy-regularized Markov decision processes and two-player games, given a generative model of the environment. SmoothCruiser makes…
Entropy-Regularized Partially Observed Markov Decision Processes
We investigate partially observed Markov decision processes (POMDPs) with cost functions regularized by entropy terms describing state, observation, and control uncertainty. Standard POMDP techniques are shown to offer b…
State EstimationA Dual Approach to Constrained Markov Decision Processes with Entropy Regularization
We study entropy-regularized constrained Markov decision processes (CMDPs) under the soft-max parameterization, in which an agent aims to maximize the entropy-regularized value function while satisfying constraints on th…
Accelerating Primal-dual Methods for Regularized Markov Decision Processes
Entropy regularized Markov decision processes have been widely used in reinforcement learning. This paper is concerned with the primal-dual formulation of the entropy regularized problems. Standard first-order methods su…
reinforcement-learningReinforcement Learning (RL)A unified view of entropy-regularized Markov decision processes
We propose a general framework for entropy-regularized average-reward reinforcement learning in Markov decision processes (MDPs). Our approach is based on extending the linear-programming formulation of policy optimizati…
Policy Gradient Methodsreinforcement-learningReinforcement LearningReinforcement Learning (RL)