Deep Episodic Memory: Encoding, Recalling, and Predicting Episodic Experiences for Robot Action Execution
We present a novel deep neural network architecture for representing robot experiences in an episodic-like memory which facilitates encoding, recalling, and predicting action experiences. Our proposed unsupervised deep episodic memory model 1) encodes observed actions in a latent vector space and, based on this latent encoding, 2) infers most similar episodes previously experienced, 3) reconstructs original episodes, and 4) predicts future frames in an end-to-end fashion. Results show that conceptually similar actions are mapped into the same region of the latent vector space. Based on these results, we introduce an action matching and retrieval mechanism, benchmark its performance on two large-scale action datasets, 20BN-something-something and ActivityNet and evaluate its generalization capability in a real-world scenario on a humanoid robot.
Code (1)
Tasks
RetrievalSimilar Papers 제목 키워드 기반
Model-Based Episodic Memory Induces Dynamic Hybrid Controls
Episodic control enables sample efficiency in reinforcement learning by recalling past experiences from an episodic memory. We propose a new model-based episodic memory of trajectories addressing current limitations of e…
modelreinforcement-learningReinforcement LearningReinforcement Learning (RL)Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
Episodic memory plays an important role in the behavior of animals and humans. It allows the accumulation of information about current state of the environment in a task-agnostic way. This episodic representation can be …
Continual Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Latent Structured Hopfield Network for Semantic Association and Retrieval
Episodic memory enables humans to recall past experiences by associating semantic elements such as objects, locations, and time into coherent event representations. While large pretrained models have shown remarkable pro…
DecoderRetrievalGENESIS: A Generative Model of Episodic-Semantic Interaction
A central challenge in cognitive neuroscience is to explain how semantic and episodic memory, two major forms of declarative memory, typically associated with cortical and hippocampal processing, interact to support lear…
Few-shot Generation via Recalling Brain-Inspired Episodic-Semantic Memory
Aimed at adapting a generative model to a novel generation task with only a few given data samples, the capability of few-shot generation is crucial for many real-world applications with limited data, \emph{e.g.}, artist…