Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay
A lifelong learning agent is able to continually learn from potentially infinite streams of pattern sensory data. One major historic difficulty in building agents that adapt in this way is that neural systems struggle to retain previously-acquired knowledge when learning from new samples. This problem is known as catastrophic forgetting (interference) and remains an unsolved problem in the domain of machine learning to this day. While forgetting in the context of feedforward networks has been examined extensively over the decades, far less has been done in the context of alternative architectures such as the venerable self-organizing map (SOM), an unsupervised neural model that is often used in tasks such as clustering and dimensionality reduction. Although the competition among its internal neurons might carry the potential to improve memory retention, we observe that a fixed-sized SOM trained on task incremental data, i.e., it receives data points related to specific classes at certain temporal increments, experiences significant forgetting. In this study, we propose the continual SOM (c-SOM), a model that is capable of reducing its own forgetting when processing information.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionLifelong learningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Saturation Self-Organizing Map
Continual learning poses a fundamental challenge for neural systems, which often suffer from catastrophic forgetting when exposed to sequential tasks. Self-Organizing Maps (SOMs), despite their interpretability and effic…
Continual LearningDendritic Self-Organizing Maps for Continual Learning
Current deep learning architectures show remarkable performance when trained in large-scale, controlled datasets. However, the predictive ability of these architectures significantly decreases when learning new classes i…
Continual LearningSplit-CIFAR-10Split-MNISTSelf-Organizing Maps with Variable Input Length for Motif Discovery and Word Segmentation
Time Series Motif Discovery (TSMD) is defined as searching for patterns that are previously unknown and appear with a given frequency in time series. Another problem strongly related with TSMD is Word Segmentation. This …
Language AcquisitionSegmentationTime SeriesTime Series AnalysisContinual Learning with Self-Organizing Maps
Despite remarkable successes achieved by modern neural networks in a wide range of applications, these networks perform best in domain-specific stationary environments where they are trained only once on large-scale cont…
Continual LearningNeuro-mimetic Task-free Unsupervised Online Learning with Continual Self-Organizing Maps
An intelligent system capable of continual learning is one that can process and extract knowledge from potentially infinitely long streams of pattern vectors. The major challenge that makes crafting such a system difficu…
class-incremental learningClass Incremental LearningContinual LearningDimensionality Reduction+2