vCLIMB: A Novel Video Class Incremental Learning Benchmark
Continual learning (CL) is under-explored in the video domain. The few existing works contain splits with imbalanced class distributions over the tasks, or study the problem in unsuitable datasets. We introduce vCLIMB, a novel video continual learning benchmark. vCLIMB is a standardized test-bed to analyze catastrophic forgetting of deep models in video continual learning. In contrast to previous work, we focus on class incremental continual learning with models trained on a sequence of disjoint tasks, and distribute the number of classes uniformly across the tasks. We perform in-depth evaluations of existing CL methods in vCLIMB, and observe two unique challenges in video data. The selection of instances to store in episodic memory is performed at the frame level. Second, untrimmed training data influences the effectiveness of frame sampling strategies. We address these two challenges by proposing a temporal consistency regularization that can be applied on top of memory-based continual learning methods. Our approach significantly improves the baseline, by up to 24% on the untrimmed continual learning task.
Code (0)
등록된 구현이 없습니다.
Tasks
class-incremental learningClass Incremental LearningContinual LearningIncremental LearningSimilar Papers 제목 키워드 기반
ESSENTIAL: Episodic and Semantic Memory Integration for Video Class-Incremental Learning
In this work, we tackle the problem of video classincremental learning (VCIL). Many existing VCIL methods mitigate catastrophic forgetting by rehearsal training with a few temporally dense samples stored in episodic memo…
class-incremental learningGeneral KnowledgeWhen Video Classification Meets Incremental Classes
With the rapid development of social media, tremendous videos with new classes are generated daily, which raise an urgent demand for video classification methods that can continuously update new classes while maintaining…
Classificationclass-incremental learningClass Incremental LearningIncremental Learning+2Learning Conditional Space-Time Prompt Distributions for Video Class-Incremental Learning
Recent advancements in prompt-based learning have significantly advanced image and video class-incremental learning. However, the prompts learned by these methods often fail to capture the diverse and informative cha…
class-incremental learningClass Incremental LearningIncremental LearningPrompt LearningVideo Domain Incremental Learning for Human Action Recognition in Home Environments
It is significantly challenging to recognize daily human actions in homes due to the diversity and dynamic changes in unconstrained home environments. It spurs the need to continually adapt to various users and scenes. F…
Action Recognitionclass-incremental learningClass Incremental LearningContinual Learning+3Learning a Condensed Frame for Memory-Efficient Video Class-Incremental Learning
Recent incremental learning for action recognition usually stores representative videos to mitigate catastrophic forgetting. However, only a few bulky videos can be stored due to the limited memory. To address this probl…
Action Recognitionclass-incremental learningClass Incremental LearningIncremental Learning