Class-incremental Learning using a Sequence of Partial Implicitly Regularized Classifiers

2021-04-04 · Sobirdzhon Bobiev, Adil Khan, Syed Muhammad Ahsan Raza Kazmi

In class-incremental learning, the objective is to learn a number of classes sequentially without having access to the whole training data. However, due to a problem known as catastrophic forgetting, neural networks suffer substantial performance drop in such settings. The problem is often approached by experience replay, a method which stores a limited number of samples to be replayed in future steps to reduce forgetting of the learned classes. When using a pretrained network as a feature extractor, we show that instead of training a single classifier incrementally, it is better to train a number of specialized classifiers which do not interfere with each other yet can cooperatively predict a single class. Our experiments on CIFAR100 dataset show that the proposed method improves the performance over SOTA by a large margin.

📄 PDF Abstract BibTeX arXiv:2104.01577

Code (0)

등록된 구현이 없습니다.

Tasks

class-incremental learning Class Incremental Learning Incremental Learning

Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU

2021-09-15 · EMNLP 2021 11 · Patrick Kahardipraja, Brielen Madureira, David Schlangen

Incremental processing allows interactive systems to respond based on partial inputs, which is a desirable property e.g. in dialogue agents. The currently popular Transformer architecture inherently processes sequences a…

On the Soft-Subnetwork for Few-shot Class Incremental Learning

2022-09-15 · Haeyong Kang, Jaehong Yoon, Sultan Rizky Hikmawan Madjid, Sung Ju Hwang 외

Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive performance of the dense network, we propo…

class-incremental learning Class Incremental Learning Few-Shot Class-Incremental Learning Incremental Learning

Computationally Budgeted Continual Learning: What Does Matter?

2023-03-20 · CVPR 2023 1 · Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet Dokania, Philip H. S. Torr 외

Continual Learning (CL) aims to sequentially train models on streams of incoming data that vary in distribution by preserving previous knowledge while adapting to new data. Current CL literature focuses on restricted acc…

Continual Learning GPU

Incremental Few-Shot Learning via Implanting and Compressing

2022-03-19 · Yiting Li, Haiyue Zhu, Xijia Feng, Zilong Cheng 외

This work focuses on tackling the challenging but realistic visual task of Incremental Few-Shot Learning (IFSL), which requires a model to continually learn novel classes from only a few examples while not forgetting the…

Few-Shot Learning image-classification Image Classification object-detection+1

An Online Sequence-to-Sequence Model Using Partial Conditioning

2016-12-01 · NeurIPS 2016 12 · Navdeep Jaitly, Quoc V. Le, Oriol Vinyals, Ilya Sutskever 외

Sequence-to-sequence models have achieved impressive results on various tasks. However, they are unsuitable for tasks that require incremental predictions to be made as more data arrives or tasks that have long input seq…