Generalization of Reinforcement Learners with Working and Episodic Memory
Memory is an important aspect of intelligence and plays a role in many deep reinforcement learning models. However, little progress has been made in understanding when specific memory systems help more than others and how well they generalize. The field also has yet to see a prevalent consistent and rigorous approach for evaluating agent performance on holdout data. In this paper, we aim to develop a comprehensive methodology to test different kinds of memory in an agent and assess how well the agent can apply what it learns in training to a holdout set that differs from the training set along dimensions that we suggest are relevant for evaluating memory-specific generalization. To that end, we first construct a diverse set of memory tasks that allow us to evaluate test-time generalization across multiple dimensions. Second, we develop and perform multiple ablations on an agent architecture that combines multiple memory systems, observe its baseline models, and investigate its performance against the task suite.
Code (1)
Tasks
Deep Reinforcement LearningHoldout SetReinforcement LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Transformers are Meta-Reinforcement Learners
The transformer architecture and variants presented a remarkable success across many machine learning tasks in recent years. This success is intrinsically related to the capability of handling long sequences and the pres…
continuous-controlContinuous ControlMeta Reinforcement LearningOut-of-Distribution Generalization+3Transformers are Meta-Reinforcement Learners
The transformer architecture and variants presented remarkable success across many machine learning tasks in recent years. This success is intrinsically related to the capability of handling long sequences and the presen…
continuous-controlContinuous ControlMeta Reinforcement LearningOut-of-Distribution Generalization+3Flexible Prefrontal Control over Hippocampal Episodic Memory for Goal-Directed Generalization
Many tasks require flexibly modifying perception and behavior based on current goals. Humans can retrieve episodic memories from days to years ago, using them to contextualize and generalize behaviors across novel but st…
HippocampusRetrievalTwo-Memory Reinforcement Learning
While deep reinforcement learning has shown important empirical success, it tends to learn relatively slow due to slow propagation of rewards information and slow update of parametric neural networks. Non-parametric epis…
Deep Reinforcement Learningreinforcement-learningReinforcement LearningRepresentation Learning+1Is prioritized sweeping the better episodic control?
Episodic control has been proposed as a third approach to reinforcement learning, besides model-free and model-based control, by analogy with the three types of human memory. i.e. episodic, procedural and semantic memory…
Reinforcement Learning