Sufficient Markov Decision Processes with Alternating Deep Neural Networks
Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with a large or indefinite time horizon. Choosing a representation of the underlying decision process that is both Markov and low-dimensional is non-trivial. We propose a method for constructing a low-dimensional representation of the original decision process for which: 1. the MDP model holds; 2. a decision strategy that maximizes mean utility when applied to the low-dimensional representation also maximizes mean utility when applied to the original process. We use a deep neural network to define a class of potential process representations and estimate the process of lowest dimension within this class. The method is illustrated using data from a mobile study on heavy drinking and smoking among college students.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
On the Expressivity of Multidimensional Markov Reward
We consider the expressivity of Markov rewards in sequential decision making under uncertainty. We view reward functions in Markov Decision Processes (MDPs) as a means to characterize desired behaviors of agents. Assumin…
Decision MakingDecision Making Under UncertaintySequential Decision MakingHomomorphic Mappings for Value-Preserving State Aggregation in Markov Decision Processes
State aggregation aims to reduce the computational complexity of solving Markov Decision Processes (MDPs) while preserving the performance of the original system. A fundamental challenge lies in optimizing policies withi…
Computational EfficiencyLinear Additive Markov Processes
We introduce LAMP: the Linear Additive Markov Process. Transitions in LAMP may be influenced by states visited in the distant history of the process, but unlike higher-order Markov processes, LAMP retains an efficient pa…
Detecting Spiky Corruption in Markov Decision Processes
Current reinforcement learning methods fail if the reward function is imperfect, i.e. if the agent observes reward different from what it actually receives. We study this problem within the formalism of Corrupt Reward Ma…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)MATE: Solving Contextual Markov Decision Processes with Memory of Accumulated Transition Embeddings
We propose MATE, a simple yet effective memory architecture for solving Contextual Markov Decision Processes (CMDPs), a family of MDPs parameterized by an unobserved context. In CMDPs, an optimal agent can adapt online b…