Episodic memory for continual model learning
Both the human brain and artificial learning agents operating in real-world or comparably complex environments are faced with the challenge of online model selection. In principle this challenge can be overcome: hierarchical Bayesian inference provides a principled method for model selection and it converges on the same posterior for both off-line (i.e. batch) and online learning. However, maintaining a parameter posterior for each model in parallel has in general an even higher memory cost than storing the entire data set and is consequently clearly unfeasible. Alternatively, maintaining only a limited set of models in memory could limit memory requirements. However, sufficient statistics for one model will usually be insufficient for fitting a different kind of model, meaning that the agent loses information with each model change. We propose that episodic memory can circumvent the challenge of limited memory-capacity online model selection by retaining a selected subset of data points. We design a method to compute the quantities necessary for model selection even when the data is discarded and only statistics of one (or few) learnt models are available. We demonstrate on a simple model that a limited-sized episodic memory buffer, when the content is optimised to retain data with statistics not matching the current representation, can resolve the fundamental challenge of online model selection.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferencemodelModel SelectionSimilar Papers 제목 키워드 기반
Continual and Multi-task Reinforcement Learning With Shared Episodic Memory
Episodic memory plays an important role in the behavior of animals and humans. It allows the accumulation of information about current state of the environment in a task-agnostic way. This episodic representation can be …
Continual Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Continual relation learning aims to continually train a model on new data to learn incessantly emerging novel relations while avoiding catastrophically forgetting old relations. Some pioneering work has proved that stori…
Continual LearningRelationSaliency-Augmented Memory Completion for Continual Learning
Continual Learning is considered a key step toward next-generation Artificial Intelligence. Among various methods, replay-based approaches that maintain and replay a small episodic memory of previous samples are one of t…
Bilevel OptimizationContinual Learningimage-classificationImage ClassificationOn Tiny Episodic Memories in Continual Learning
In continual learning (CL), an agent learns from a stream of tasks leveraging prior experience to transfer knowledge to future tasks. It is an ideal framework to decrease the amount of supervision in the existing learnin…
Class Incremental LearningContinual LearningTransfer LearningNonconvex Continual Learning with Episodic Memory
Continual learning aims to prevent catastrophic forgetting while learning a new task without accessing data of previously learned tasks. The memory for such learning scenarios build a small subset of the data for previo…
Continual Learningimage-classificationImage Classification