Blockwise Sequential Model Learning for Partially Observable Reinforcement Learning
This paper proposes a new sequential model learning architecture to solve partially observable Markov decision problems. Rather than compressing sequential information at every timestep as in conventional recurrent neural network-based methods, the proposed architecture generates a latent variable in each data block with a length of multiple timesteps and passes the most relevant information to the next block for policy optimization. The proposed blockwise sequential model is implemented based on self-attention, making the model capable of detailed sequential learning in partial observable settings. The proposed model builds an additional learning network to efficiently implement gradient estimation by using self-normalized importance sampling, which does not require the complex blockwise input data reconstruction in the model learning. Numerical results show that the proposed method significantly outperforms previous methods in various partially observable environments.
Code (1)
Tasks
Partially Observable Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
On Improving Deep Reinforcement Learning for POMDPs
Deep Reinforcement Learning (RL) recently emerged as one of the most competitive approaches for learning in sequential decision making problems with fully observable environments, e.g., computer Go. However, very little …
Atari GamesDecision MakingDeep Reinforcement Learningreinforcement-learning+5On Improving Deep Reinforcement Learning for POMDPs
Deep Reinforcement Learning (RL) recently emerged as one of the most competitive approaches for learning in sequential decision making problems with fully observable environments, e.g., computer Go. However, very little …
Atari GamesDecision MakingDeep Reinforcement Learningreinforcement-learning+5The Partially Observable History Process
We introduce the partially observable history process (POHP) formalism for reinforcement learning. POHP centers around the actions and observations of a single agent and abstracts away the presence of other players witho…
Formreinforcement-learningReinforcement LearningReinforcement Learning (RL)Leveraging Fully Observable Policies for Learning under Partial Observability
Reinforcement learning in partially observable domains is challenging due to the lack of observable state information. Thankfully, learning offline in a simulator with such state information is often possible. In particu…
Partially Observable Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Dynamic deep-reinforcement-learning algorithm in Partially Observed Markov Decision Processes
Reinforcement learning has been greatly improved in recent studies and an increased interest in real-world implementation has emerged in recent years. In many cases, due to the non-static disturbances, it becomes challen…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning