Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition
This paper presents a study on improving human action recognition through the utilization of knowledge distillation, and the combination of CNN and ViT models. The research aims to enhance the performance and efficiency of smaller student models by transferring knowledge from larger teacher models. The proposed method employs a Transformer vision network as the student model, while a convolutional network serves as the teacher model. The teacher model extracts local image features, whereas the student model focuses on global features using an attention mechanism. The Vision Transformer (ViT) architecture is introduced as a robust framework for capturing global dependencies in images. Additionally, advanced variants of ViT, namely PVT, Convit, MVIT, Swin Transformer, and Twins, are discussed, highlighting their contributions to computer vision tasks. The ConvNeXt model is introduced as a teacher model, known for its efficiency and effectiveness in computer vision. The paper presents performance results for human action recognition on the Stanford 40 dataset, comparing the accuracy and mAP of student models trained with and without knowledge distillation. The findings illustrate that the suggested approach significantly improves the accuracy and mAP when compared to training networks under regular settings. These findings emphasize the potential of combining local and global features in action recognition tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionKnowledge DistillationTemporal Action LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Vi-LAD: Vision-Language Attention Distillation for Socially-Aware Robot Navigation in Dynamic Environments
We introduce Vision-Language Attention Distillation (Vi-LAD), a novel approach for distilling socially compliant navigation knowledge from a large Vision-Language Model (VLM) into a lightweight transformer model for real…
Knowledge DistillationMotion PlanningRobot NavigationDistilling Inductive Bias: Knowledge Distillation Beyond Model Compression
With the rapid development of computer vision, Vision Transformers (ViTs) offer the tantalizing prospect of unified information processing across visual and textual domains. But due to the lack of inherent inductive bias…
Inductive BiasKnowledge DistillationmodelModel CompressionLoReTrack: Efficient and Accurate Low-Resolution Transformer Tracking
High-performance Transformer trackers have shown excellent results, yet they often bear a heavy computational load. Observing that a smaller input can immediately and conveniently reduce computations without changing the…
CPUKnowledge DistillationVisual Object TrackingDistilling HuBERT with LSTMs via Decoupled Knowledge Distillation
Much research effort is being applied to the task of compressing the knowledge of self-supervised models, which are powerful, yet large and memory consuming. In this work, we show that the original method of knowledge di…
Automatic Speech RecognitionKnowledge Distillationspeech-recognitionSpeech RecognitionDistilling Token-Pruned Pose Transformer for 2D Human Pose Estimation
Human pose estimation has seen widespread use of transformer models in recent years. Pose transformers benefit from the self-attention map, which captures the correlation between human joint tokens and the image. However…
2D Human Pose EstimationPose Estimation