MoLo: Motion-augmented Long-short Contrastive Learning for Few-shot Action Recognition
Current state-of-the-art approaches for few-shot action recognition achieve promising performance by conducting frame-level matching on learned visual features. However, they generally suffer from two limitations: i) the matching procedure between local frames tends to be inaccurate due to the lack of guidance to force long-range temporal perception; ii) explicit motion learning is usually ignored, leading to partial information loss. To address these issues, we develop a Motion-augmented Long-short Contrastive Learning (MoLo) method that contains two crucial components, including a long-short contrastive objective and a motion autodecoder. Specifically, the long-short contrastive objective is to endow local frame features with long-form temporal awareness by maximizing their agreement with the global token of videos belonging to the same class. The motion autodecoder is a lightweight architecture to reconstruct pixel motions from the differential features, which explicitly embeds the network with motion dynamics. By this means, MoLo can simultaneously learn long-range temporal context and motion cues for comprehensive few-shot matching. To demonstrate the effectiveness, we evaluate MoLo on five standard benchmarks, and the results show that MoLo favorably outperforms recent advanced methods. The source code is available at https://github.com/alibaba-mmai-research/MoLo.
Code (1)
Tasks
Action RecognitionContrastive LearningFew-Shot action recognitionFew Shot Action RecognitionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MotionRAG-Diff: A Retrieval-Augmented Diffusion Framework for Long-Term Music-to-Dance Generation
Generating long-term, coherent, and realistic music-conditioned dance sequences remains a challenging task in human motion synthesis. Existing approaches exhibit critical limitations: motion graph methods rely on fixed t…
Contrastive LearningMotion SynthesisRAGRetrieval+2NCL: Textual Backdoor Defense Using Noise-augmented Contrastive Learning
At present, backdoor attacks attract attention as they do great harm to deep learning models. The adversary poisons the training data making the model being injected with a backdoor after being trained unconsciously by v…
backdoor defenseContrastive LearningUniversal Coefficients and Mayer-Vietoris for Moore Homology of Ample Groupoids
We establish two structural results for Moore homology of ample groupoids. First, for every ample groupoid $\mathcal{G}$ and every discrete abelian coefficient group $A$, we prove a universal coefficient theorem relating…
Motion-aware Contrastive Video Representation Learning via Foreground-background Merging
In light of the success of contrastive learning in the image domain, current self-supervised video representation learning methods usually employ contrastive loss to facilitate video representation learning. When naively…
Action RecognitionContrastive LearningRepresentation LearningMulti-OphthaLingua: A Multilingual Benchmark for Assessing and Debiasing LLM Ophthalmological QA in LMICs
Current ophthalmology clinical workflows are plagued by over-referrals, long waits, and complex and heterogeneous medical records. Large language models (LLMs) present a promising solution to automate various procedures …
Question AnsweringRAGRetrievalRetrieval-augmented Generation+1