Of Cores: A Partial-Exploration Framework for Markov Decision Processes
We introduce a framework for approximate analysis of Markov decision processes (MDP) with bounded-, unbounded-, and infinite-horizon properties. The main idea is to identify a "core" of an MDP, i.e., a subsystem where we provably remain with high probability, and to avoid computation on the less relevant rest of the state space. Although we identify the core using simulations and statistical techniques, it allows for rigorous error bounds in the analysis. Consequently, we obtain efficient analysis algorithms based on partial exploration for various settings, including the challenging case of strongly connected systems.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Provable Representation with Efficient Planning for Partial Observable Reinforcement Learning
In most real-world reinforcement learning applications, state information is only partially observable, which breaks the Markov decision process assumption and leads to inferior performance for algorithms that conflate o…
Partially Observable Reinforcement Learningreinforcement-learningReinforcement LearningSequential Monte Carlo for Policy Optimization in Continuous POMDPs
Optimal decision-making under partial observability requires agents to balance reducing uncertainty (exploration) against pursuing immediate objectives (exploitation). In this paper, we introduce a novel policy optimizat…
Decision MakingDensity-Based Bonuses on Learned Representations for Reward-Free Exploration in Deep Reinforcement Learning
In this paper, we study the problem of representation learning and exploration in reinforcement learning. We propose a framework to compute exploration bonuses based on density estimation, that can be used with any repre…
Deep Reinforcement LearningDensity Estimationreinforcement-learningReinforcement Learning (RL)+1GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond
We study sample efficient reinforcement learning (RL) under the general framework of interactive decision making, which includes Markov decision process (MDP), partially observable Markov decision process (POMDP), and pr…
Decision MakingReinforcement Learning (RL)Beyond Optimism: Exploration With Partially Observable Rewards
Exploration in reinforcement learning (RL) remains an open challenge. RL algorithms rely on observing rewards to train the agent, and if informative rewards are sparse the agent learns slowly or may not learn at all. To …
BenchmarkingReinforcement Learning (RL)