Explaining Robustness to Catastrophic Forgetting Through Incremental Concept Formation

2025-10-27 · Nicki Barari, Edward Kim, Christopher MacLellan arxiv

Catastrophic forgetting remains a central challenge in continual learning, where models are required to integrate new knowledge over time without losing what they have previously learned. In prior work, we introduced Cobweb/4V, a hierarchical concept formation model that exhibited robustness to catastrophic forgetting in visual domains. Motivated by this robustness, we examine three hypotheses regarding the factors that contribute to such stability: (1) adaptive structural reorganization enhances knowledge retention, (2) sparse and selective updates reduce interference, and (3) information-theoretic learning based on sufficiency statistics provides advantages over gradient-based backpropagation. To test these hypotheses, we compare Cobweb/4V with neural baselines, including CobwebNN, a neural implementation of the Cobweb framework introduced in this work. Experiments on datasets of varying complexity (MNIST, Fashion-MNIST, MedMNIST, and CIFAR-10) show that adaptive restructuring enhances learning plasticity, sparse updates help mitigate interference, and the information-theoretic learning process preserves prior knowledge without revisiting past data. Together, these findings provide insight into mechanisms that can mitigate catastrophic forgetting and highlight the potential of concept-based, information-theoretic approaches for building stable and adaptive continual learning systems.

📄 PDF Abstract BibTeX arXiv:2510.23756

Code (0)

등록된 구현이 없습니다.

Tasks

Continual Learning

Demystifying Catastrophic Forgetting in Two-Stage Incremental Object Detector

2025-02-08 · Qirui Wu, Shizhou Zhang, De Cheng, Yinghui Xing 외

Catastrophic forgetting is a critical chanllenge for incremental object detection (IOD). Most existing methods treat the detector monolithically, relying on instance replay or knowledge distillation without analyzing com…

Incremental Learning Knowledge Distillation object-detection Object Detection

REMIND Your Neural Network to Prevent Catastrophic Forgetting

2019-10-06 · ECCV 2020 8 · Tyler L. Hayes, Kushal Kafle, Robik Shrestha, Manoj Acharya 외

People learn throughout life. However, incrementally updating conventional neural networks leads to catastrophic forgetting. A common remedy is replay, which is inspired by how the brain consolidates memory. Replay invol…

Quantization Question Answering Visual Question Answering Visual Question Answering (VQA)

Incremental Class Learning using Variational Autoencoders with Similarity Learning

2021-10-04 · Jiahao Huo, Terence L. Van Zyl

Catastrophic forgetting in neural networks during incremental learning remains a challenging problem. Previous research investigated catastrophic forgetting in fully connected networks, with some earlier work exploring a…

Incremental Learning Metric Learning Triplet

How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation

2021-02-04 · Ekdeep Singh Lubana, Puja Trivedi, Danai Koutra, Robert P. Dick

Catastrophic forgetting undermines the effectiveness of deep neural networks (DNNs) in scenarios such as continual learning and lifelong learning. While several methods have been proposed to tackle this problem, there is…

Continual Learning Lifelong learning

SupportNet: solving catastrophic forgetting in class incremental learning with support data

2018-06-08 · Yu Li, Zhongxiao Li, Lizhong Ding, Yijie Pan 외

A plain well-trained deep learning model often does not have the ability to learn new knowledge without forgetting the previously learned knowledge, which is known as catastrophic forgetting. Here we propose a novel meth…

class-incremental learning Class Incremental Learning Deep Learning Incremental Learning