Continual Learning and Catastrophic Forgetting
This book chapter delves into the dynamics of continual learning, which is the process of incrementally learning from a non-stationary stream of data. Although continual learning is a natural skill for the human brain, it is very challenging for artificial neural networks. An important reason is that, when learning something new, these networks tend to quickly and drastically forget what they had learned before, a phenomenon known as catastrophic forgetting. Especially in the last decade, continual learning has become an extensively studied topic in deep learning. This book chapter reviews the insights that this field has generated.
Code (0)
등록된 구현이 없습니다.
Tasks
Continual LearningSimilar Papers 제목 키워드 기반
Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting
Addressing catastrophic forgetting is one of the key challenges in continual learning where machine learning systems are trained with sequential or streaming tasks. Despite recent remarkable progress in state-of-the-art …
Continual LearningNeural Architecture Searchparameter estimationPermuted-MNISTWide Neural Networks Forget Less Catastrophically
A primary focus area in continual learning research is alleviating the "catastrophic forgetting" problem in neural networks by designing new algorithms that are more robust to the distribution shifts. While the recent pr…
Continual LearningOvercoming Catastrophic Forgetting by XAI
Explaining the behaviors of deep neural networks, usually considered as black boxes, is critical especially when they are now being adopted over diverse aspects of human life. Taking the advantages of interpretable machi…
Continual LearningExplainable Artificial Intelligence (XAI)Interpretable Machine LearningTechnical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners
In the SSLAD-Track 3B challenge on continual learning, we propose the method of COntinual Learning with Transformer (COLT). We find that transformers suffer less from catastrophic forgetting compared to convolutional neu…
Continual LearningKnowledge Distillationobject-detectionObject DetectionUnderstanding Catastrophic Forgetting and Remembering in Continual Learning with Optimal Relevance Mapping
Catastrophic forgetting in neural networks is a significant problem for continual learning. A majority of the current methods replay previous data during training, which violates the constraints of an ideal continual lea…
Continual Learning