Rebalancing Multi-Label Class-Incremental Learning

2024-08-22 · Kaile Du, Yifan Zhou, Fan Lyu, Yuyang Li, Junzhou Xie, Yixi Shen, Fuyuan Hu, Guangcan Liu

Multi-label class-incremental learning (MLCIL) is essential for real-world multi-label applications, allowing models to learn new labels while retaining previously learned knowledge continuously. However, recent MLCIL approaches can only achieve suboptimal performance due to the oversight of the positive-negative imbalance problem, which manifests at both the label and loss levels because of the task-level partial label issue. The imbalance at the label level arises from the substantial absence of negative labels, while the imbalance at the loss level stems from the asymmetric contributions of the positive and negative loss parts to the optimization. To address the issue above, we propose a Rebalance framework for both the Loss and Label levels (RebLL), which integrates two key modules: asymmetric knowledge distillation (AKD) and online relabeling (OR). AKD is proposed to rebalance at the loss level by emphasizing the negative label learning in classification loss and down-weighting the contribution of overconfident predictions in distillation loss. OR is designed for label rebalance, which restores the original class distribution in memory by online relabeling the missing classes. Our comprehensive experiments on the PASCAL VOC and MS-COCO datasets demonstrate that this rebalancing strategy significantly improves performance, achieving new state-of-the-art results even with a vanilla CNN backbone.

📄 PDF Abstract BibTeX arXiv:2408.12161

Code (0)

등록된 구현이 없습니다.

Tasks

class-incremental learning Class Incremental Learning Incremental Learning Knowledge Distillation

Methods 이 논문이 사용한 방법론

Knowledge Distillation A very simple way to improve the performance of almost any machine learning algorithm is to train many different models on the same data and then to average their predictions.…

CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning

2021-02-18 · CVPR 2021 1 · Chen Wei, Kihyuk Sohn, Clayton Mellina, Alan Yuille 외

Semi-supervised learning on class-imbalanced data, although a realistic problem, has been under studied. While existing semi-supervised learning (SSL) methods are known to perform poorly on minority classes, we find that…

Long-tailed Recognition with Model Rebalancing

2025-10-09 · Jiaan Luo, Feng Hong, Qiang Hu, Xiaofeng Cao 외 arxiv

Long-tailed recognition is ubiquitous and challenging in deep learning and even in the downstream finetuning of foundation models, since the skew class distribution generally prevents the model generalization to the tail…

Data Augmentation

Semi-Supervised Object Detection with Adaptive Class-Rebalancing Self-Training

2021-07-11 · Fangyuan Zhang, Tianxiang Pan, Bin Wang

This study delves into semi-supervised object detection (SSOD) to improve detector performance with additional unlabeled data. State-of-the-art SSOD performance has been achieved recently by self-training, in which train…

object-detection Object Detection Semi-Supervised Object Detection

Long-Tailed Partial Label Learning via Dynamic Rebalancing

2023-02-10 · Feng Hong, Jiangchao Yao, Zhihan Zhou, Ya zhang 외

Real-world data usually couples the label ambiguity and heavy imbalance, challenging the algorithmic robustness of partial label learning (PLL) and long-tailed learning (LT). The straightforward combination of LT and PLL…

Partial Label Learning

Learning a Unified Classifier Incrementally via Rebalancing

2019-06-01 · CVPR 2019 6 · Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang 외

Conventionally, deep neural networks are trained offline, relying on a large dataset prepared in advance. This paradigm is often challenged in real-world applications, e.g. online services that involve continuous streams…

Class Incremental Learning Incremental Learning