Unlimited Knowledge Distillation for Action Recognition in the Dark
Dark videos often lose essential information, which causes the knowledge learned by networks is not enough to accurately recognize actions. Existing knowledge assembling methods require massive GPU memory to distill the knowledge from multiple teacher models into a student model. In action recognition, this drawback becomes serious due to much computation required by video process. Constrained by limited computation source, these approaches are infeasible. To address this issue, we propose an unlimited knowledge distillation (UKD) in this paper. Compared with existing knowledge assembling methods, our UKD can effectively assemble different knowledge without introducing high GPU memory consumption. Thus, the number of teaching models for distillation is unlimited. With our UKD, the network's learned knowledge can be remarkably enriched. Our experiments show that the single stream network distilled with our UKD even surpasses a two-stream network. Extensive experiments are conducted on the ARID dataset.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionGPUKnowledge DistillationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
DL-KDD: Dual-Light Knowledge Distillation for Action Recognition in the Dark
Human action recognition in dark videos is a challenging task for computer vision. Recent research focuses on applying dark enhancement methods to improve the visibility of the video. However, such video processing resul…
Action RecognitionKnowledge DistillationTemporal Action LocalizationVideo ClassificationARID: A New Dataset for Recognizing Action in the Dark
The task of action recognition in dark videos is useful in various scenarios, e.g., night surveillance and self-driving at night. Though progress has been made in the action recognition task for videos in normal illumina…
Action RecognitionDark Transformer: A Video Transformer for Action Recognition in the Dark
Recognizing human actions in adverse lighting conditions presents significant challenges in computer vision, with wide-ranging applications in visual surveillance and nighttime driving. Existing methods tackle action rec…
Action ClassificationAction RecognitionAdam: Dense Retrieval Distillation with Adaptive Dark Examples
To improve the performance of the dual-encoder retriever, one effective approach is knowledge distillation from the cross-encoder ranker. Existing works construct the candidate passages following the supervised learning …
Knowledge DistillationRetrievalWhat Do Students Learn? A Feature-Level Analysis of Dark Knowledge
Knowledge Distillation (KD) is a powerful tool for model compression, yet the precise mechanisms by which student models acquire feature representations remain underexplored. In this work, we analyze student feature lear…
Knowledge DistillationModel Compression