Depth Guided Adaptive Meta-Fusion Network for Few-shot Video Recognition
Humans can easily recognize actions with only a few examples given, while the existing video recognition models still heavily rely on the large-scale labeled data inputs. This observation has motivated an increasing interest in few-shot video action recognition, which aims at learning new actions with only very few labeled samples. In this paper, we propose a depth guided Adaptive Meta-Fusion Network for few-shot video recognition which is termed as AMeFu-Net. Concretely, we tackle the few-shot recognition problem from three aspects: firstly, we alleviate this extremely data-scarce problem by introducing depth information as a carrier of the scene, which will bring extra visual information to our model; secondly, we fuse the representation of original RGB clips with multiple non-strictly corresponding depth clips sampled by our temporal asynchronization augmentation mechanism, which synthesizes new instances at feature-level; thirdly, a novel Depth Guided Adaptive Instance Normalization (DGAdaIN) fusion module is proposed to fuse the two-stream modalities efficiently. Additionally, to better mimic the few-shot recognition process, our model is trained in the meta-learning way. Extensive experiments on several action recognition benchmarks demonstrate the effectiveness of our model.
Code (1)
Tasks
Action RecognitionFew Shot Action RecognitionMeta-LearningTemporal Action LocalizationVideo RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
Depth completion upgrades sparse depth measurements into dense depth maps guided by a conventional image. Existing methods for this highly ill-posed task operate in tightly constrained settings and tend to struggle when …
DenoisingDepth CompletionDepth EstimationMonocular Depth Estimation+1Adaptive Multi-Scale Correlation Meta-Network for Few-Shot Remote Sensing Image Classification
Few-shot learning in remote sensing remains challenging due to three factors: the scarcity of labeled data, substantial domain shifts, and the multi-scale nature of geospatial objects. To address these issues, we introdu…
Remote Sensing Image ClassificationFew-Shot LearningMeta Guided Metric Learner for Overcoming Class Confusion in Few-Shot Road Object Detection
Localization and recognition of less-occurring road objects have been a challenge in autonomous driving applications due to the scarcity of data samples. Few-Shot Object Detection techniques extend the knowledge from exi…
Autonomous DrivingFew-Shot Object DetectionMetric LearningObject+2IGAF: Incremental Guided Attention Fusion for Depth Super-Resolution
Accurate depth estimation is crucial for many fields, including robotics, navigation, and medical imaging. However, conventional depth sensors often produce low-resolution (LR) depth maps, making detailed scene perceptio…
Depth EstimationSensor FusionSuper-ResolutionAuraFusion360: Augmented Unseen Region Alignment for Reference-based 360° Unbounded Scene Inpainting
Three-dimensional scene inpainting is crucial for applications from virtual reality to architectural visualization, yet existing methods struggle with view consistency and geometric accuracy in 360{\deg} unbounded scenes…