Anytime Continual Learning for Open Vocabulary Classification
We propose an approach for anytime continual learning (AnytimeCL) for open vocabulary image classification. The AnytimeCL problem aims to break away from batch training and rigid models by requiring that a system can predict any set of labels at any time and efficiently update and improve when receiving one or more training samples at any time. Despite the challenging goal, we achieve substantial improvements over recent methods. We propose a dynamic weighting between predictions of a partially fine-tuned model and a fixed open vocabulary model that enables continual improvement when training samples are available for a subset of a task's labels. We also propose an attention-weighted PCA compression of training features that reduces storage and computation with little impact to model accuracy. Our methods are validated with experiments that test flexibility of learning and inference. Code is available at https://github.com/jessemelpolio/AnytimeCL.
Code (1)
Tasks
ClassificationContinual Learningimage-classificationImage ClassificationOpen Vocabulary Image ClassificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Continual Learning in Open-vocabulary Classification with Complementary Memory Systems
We introduce a method for flexible and efficient continual learning in open-vocabulary image classification, drawing inspiration from the complementary learning systems observed in human cognition. Specifically, we propo…
Continual Learningimage-classificationImage ClassificationOpen Vocabulary Image ClassificationkNN-CLIP: Retrieval Enables Training-Free Segmentation on Continually Expanding Large Vocabularies
Continual segmentation has not yet tackled the challenge of improving open-vocabulary segmentation models with training data for accurate segmentation across large, continually expanding vocabularies. We discover that tr…
Panoptic SegmentationRetrievalSegmentationZero Shot SegmentationAnytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging
Large language models are increasingly trained in continual or open-ended settings, where the total training horizon is not known in advance. Despite this, most existing pretraining recipes are not anytime: they rely on …
COVD: Continual Open-Vocabulary Object Detection with Novel Concept Injection
Open-vocabulary object detection (OVD) has made significant progress, enabling detectors to generalize from seen to unseen categories. However, real-world category spaces continually evolve, and existing OVD models still…
Continual LearningObject DetectionCL-CLIP: CLIP-Based Continual Learning Framework with Cost-Volume Category Decoupling for Object Detection
Continual Object Detection (COD) requires a detector to acquire new categories over time while preserving previously learned ones. This goal is closely related to open-vocabulary detection, since both settings require re…
Zero-shot GeneralizationContinual LearningObject Detection