RepCL: Exploring Effective Representation for Continual Text Classification

2023-05-12 · YiFan Song, Peiyi Wang, Dawei Zhu, Tianyu Liu, Zhifang Sui, Sujian Li

Continual learning (CL) aims to constantly learn new knowledge over time while avoiding catastrophic forgetting on old tasks. In this work, we focus on continual text classification under the class-incremental setting. Recent CL studies find that the representations learned in one task may not be effective for other tasks, namely representation bias problem. For the first time we formally analyze representation bias from an information bottleneck perspective and suggest that exploiting representations with more class-relevant information could alleviate the bias. To this end, we propose a novel replay-based continual text classification method, RepCL. Our approach utilizes contrastive and generative representation learning objectives to capture more class-relevant features. In addition, RepCL introduces an adversarial replay strategy to alleviate the overfitting problem of replay. Experiments demonstrate that RepCL effectively alleviates forgetting and achieves state-of-the-art performance on three text classification tasks.

📄 PDF Abstract BibTeX arXiv:2305.07289

Code (0)

등록된 구현이 없습니다.

Tasks

Classification Continual Learning Representation Learning text-classification Text Classification

BRepCLIP: Contrastive Multimodal Pretraining on BRep Primitives for CAD Understanding

2026-06-03 · Muhammad Usama, Didier Stricker, Mohammad Sadil Khan, Muhammad Zeshan Afzal arxiv

Learning representations of CAD models is a largely open problem. While 3D representation learning has flourished around point clouds and meshes, the native format of CAD - boundary representations BReps, which encodes e…

Representation Learning Point Clouds

PILOT: A Pre-Trained Model-Based Continual Learning Toolbox

2023-09-13 · Hai-Long Sun, Da-Wei Zhou, Han-Jia Ye, De-Chuan Zhan

While traditional machine learning can effectively tackle a wide range of problems, it primarily operates within a closed-world setting, which presents limitations when dealing with streaming data. As a solution, increme…

class-incremental learning Class Incremental Learning Continual Learning Incremental Learning

Mask and Compress: Efficient Skeleton-based Action Recognition in Continual Learning

2024-07-01 · Matteo Mosconi, Andriy Sorokin, Aniello Panariello, Angelo Porrello 외

The use of skeletal data allows deep learning models to perform action recognition efficiently and effectively. Herein, we believe that exploring this problem within the context of Continual Learning is crucial. While nu…

Action Recognition Continual Learning Human Activity Recognition Skeleton Based Action Recognition+1

Exploring Multimodal Prompts For Unsupervised Continuous Anomaly Detection

2026-03-23 · Mingle Zhou, Jiahui Liu, Jin Wan, Gang Li 외 arxiv

Unsupervised Continuous Anomaly Detection (UCAD) is gaining attention for effectively addressing the catastrophic forgetting and heavy computational burden issues in traditional Unsupervised Anomaly Detection (UAD). Howe…

Unsupervised Anomaly Detection Adversarial Robustness

Continual Learning for Text Classification with Information Disentanglement Based Regularization

2021-04-12 · NAACL 2021 4 · Yufan Huang, Yanzhe Zhang, Jiaao Chen, Xuezhi Wang 외

Continual learning has become increasingly important as it enables NLP models to constantly learn and gain knowledge over time. Previous continual learning methods are mainly designed to preserve knowledge from previous …

Continual Learning Disentanglement General Classification Sentence+2