Variational Recurrent Models for Solving Partially Observable Control Tasks
In partially observable (PO) environments, deep reinforcement learning (RL) agents often suffer from unsatisfactory performance, since two problems need to be tackled together: how to extract information from the raw observations to solve the task, and how to improve the policy. In this study, we propose an RL algorithm for solving PO tasks. Our method comprises two parts: a variational recurrent model (VRM) for modeling the environment, and an RL controller that has access to both the environment and the VRM. The proposed algorithm was tested in two types of PO robotic control tasks, those in which either coordinates or velocities were not observable and those that require long-term memorization. Our experiments show that the proposed algorithm achieved better data efficiency and/or learned more optimal policy than other alternative approaches in tasks in which unobserved states cannot be inferred from raw observations in a simple manner.
Code (1)
Tasks
Deep Reinforcement LearningMemorizationReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Real-Time Recurrent Reinforcement Learning
We introduce a biologically plausible RL framework for solving tasks in partially observable Markov decision processes (POMDPs). The proposed algorithm combines three integral parts: (1) A Meta-RL architecture, resemblin…
continuous-controlContinuous ControlPartially Observable Reinforcement Learningreinforcement-learning+1Recurrent networks, hidden states and beliefs in partially observable environments
Reinforcement learning aims to learn optimal policies from interaction with environments whose dynamics are unknown. Many methods rely on the approximation of a value function to derive near-optimal policies. In partiall…
SVQN: Sequential Variational Soft Q-Learning Networks
Partially Observable Markov Decision Processes (POMDPs) are popular and flexible models for real-world decision-making applications that demand the information from past observations to make optimal decisions. Standard r…
Decision MakingQ-Learningreinforcement-learningReinforcement Learning+1ODE-based Recurrent Model-free Reinforcement Learning for POMDPs
Neural ordinary differential equations (ODEs) are widely recognized as the standard for modeling physical mechanisms, which help to perform approximate inference in unknown physical or biological environments. In partial…
continuous-controlContinuous Controlreinforcement-learningReinforcement Learning+2Rethinking Transformers in Solving POMDPs
Sequential decision-making algorithms such as reinforcement learning (RL) in real-world scenarios inevitably face environments with partial observability. This paper scrutinizes the effectiveness of a popular architectur…
Decision MakingReinforcement Learning (RL)Sequential Decision Making