Overcoming Catastrophic Forgetting by Generative Regularization

2019-12-03 · Patrick H. Chen, Wei Wei, Cho-Jui Hsieh, Bo Dai

In this paper, we propose a new method to overcome catastrophic forgetting by adding generative regularization to Bayesian inference framework. Bayesian method provides a general framework for continual learning. We could further construct a generative regularization term for all given classification models by leveraging energy-based models and Langevin-dynamic sampling to enrich the features learned in each task. By combining discriminative and generative loss together, we empirically show that the proposed method outperforms state-of-the-art methods on a variety of tasks, avoiding catastrophic forgetting in continual learning. In particular, the proposed method outperforms baseline methods over 15% on the Fashion-MNIST dataset and 10% on the CUB dataset

📄 PDF Abstract BibTeX arXiv:1912.01238

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian Inference Continual Learning

Overcoming catastrophic forgetting in neural networks

2025-07-14 · Brandon Shuen Yi Loke, Filippo Quadri, Gabriel Vivanco, Maximilian Casagrande 외

Catastrophic forgetting is the primary challenge that hinders continual learning, which refers to a neural network ability to sequentially learn multiple tasks while retaining previously acquired knowledge. Elastic Weigh…

Continual Learning L2 Regularization Lifelong learning

SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning

2020-05-24 · Chi-Chang Lee, Yu-Chen Lin, Hsuan-Tien Lin, Hsin-Min Wang 외

Numerous noise adaptation techniques have been proposed to fine-tune deep-learning models in speech enhancement (SE) for mismatched noise environments. Nevertheless, adaptation to a new environment may lead to catastroph…

Incremental Learning Speech Enhancement

Task Confusion and Catastrophic Forgetting in Class-Incremental Learning: A Mathematical Framework for Discriminative and Generative Modelings

2024-10-28 · Milad Khademi Nori, Il-Min Kim

In class-incremental learning (class-IL), models must classify all previously seen classes at test time without task-IDs, leading to task confusion. Despite being a key challenge, task confusion lacks a theoretical under…

class-incremental learning Class Incremental Learning Incremental Learning

Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting

2018-05-20 · NeurIPS 2018 12 · Hippolyt Ritter, Aleksandar Botev, David Barber

We introduce the Kronecker factored online Laplace approximation for overcoming catastrophic forgetting in neural networks. The method is grounded in a Bayesian online learning framework, where we recursively approximate…

Permuted-MNIST

Provable Continual Learning via Sketched Jacobian Approximations

2021-12-09 · Reinhard Heckel

An important problem in machine learning is the ability to learn tasks in a sequential manner. If trained with standard first-order methods most models forget previously learned tasks when trained on a new task, which is…

Continual Learning