Imbalanced Continual Learning with Partitioning Reservoir Sampling
Continual learning from a sequential stream of data is a crucial challenge for machine learning research. Most studies have been conducted on this topic under the single-label classification setting along with an assumption of balanced label distribution. This work expands this research horizon towards multi-label classification. In doing so, we identify unanticipated adversity innately existent in many multi-label datasets, the long-tailed distribution. We jointly address the two independently solved problems, Catastropic Forgetting and the long-tailed label distribution by first empirically showing a new challenge of destructive forgetting of the minority concepts on the tail. Then, we curate two benchmark datasets, COCOseq and NUS-WIDEseq, that allow the study of both intra- and inter-task imbalances. Lastly, we propose a new sampling strategy for replay-based approach named Partitioning Reservoir Sampling (PRS), which allows the model to maintain a balanced knowledge of both head and tail classes. We publicly release the dataset and the code in our project page.
Code (1)
Tasks
Continual LearningMulti-Label ClassificationMUlTI-LABEL-ClASSIFICATIONSimilar Papers 제목 키워드 기반
Online Continual Learning from Imbalanced Data
A well-documented weakness of neural networks is the fact that they suffer from catastrophic forgetting when trained on data provided by a non-stationary distribution. Recent work in the field of continual learning attem…
Computational EfficiencyContinual LearningD-CBRS: Accounting For Intra-Class Diversity in Continual Learning
Continual learning -- accumulating knowledge from a sequence of learning experiences -- is an important yet challenging problem. In this paradigm, the model's performance for previously encountered instances may substant…
Continual LearningDiversityManagementPrincipal Gradient Direction and Confidence Reservoir Sampling for Continual Learning
Task-free online continual learning aims to alleviate catastrophic forgetting of the learner on a non-iid data stream. Experience Replay (ER) is a SOTA continual learning method, which is broadly used as the backbone alg…
Continual LearningInformation-theoretic Online Memory Selection for Continual Learning
A challenging problem in task-free continual learning is the online selection of a representative replay memory from data streams. In this work, we investigate the online memory selection problem from an information-theo…
Continual LearningGradient-matching coresets for continual learning
We devise a coreset selection method based on the idea of gradient matching: The gradients induced by the coreset should match, as closely as possible, those induced by the original training dataset. We evaluate the meth…
Continual Learning