Trajectory-aligned Space-time Tokens for Few-shot Action Recognition
We propose a simple yet effective approach for few-shot action recognition, emphasizing the disentanglement of motion and appearance representations. By harnessing recent progress in tracking, specifically point trajectories and self-supervised representation learning, we build trajectory-aligned tokens (TATs) that capture motion and appearance information. This approach significantly reduces the data requirements while retaining essential information. To process these representations, we use a Masked Space-time Transformer that effectively learns to aggregate information to facilitate few-shot action recognition. We demonstrate state-of-the-art results on few-shot action recognition across multiple datasets. Our project page is available at https://www.cs.umd.edu/~pulkit/tats
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionDisentanglementFew-Shot action recognitionFew Shot Action RecognitionRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognition
Video understanding requires effective modeling of both motion and appearance information, particularly for few-shot action recognition. While recent advances in point tracking have been shown to improve few-shot action …
Action RecognitionPoint TrackingKnowledge-Enhanced Dual-stream Zero-shot Composed Image Retrieval
We study the zero-shot Composed Image Retrieval (ZS-CIR) task, which is to retrieve the target image given a reference image and a description without training on the triplet datasets. Previous works generate pseudo-word…
AttributeImage RetrievalRetrievalTriplet+1ReflectDrive-2: Reinforcement-Learning-Aligned Self-Editing for Discrete Diffusion Driving
We introduce ReflectDrive-2, a masked discrete diffusion planner with separate action expert for autonomous driving that represents plans as discrete trajectory tokens and generates them through parallel masked decoding.…
Reinforcement LearningAutonomous DrivingHow You Move Tells What You'll Do: Trajectory-Conditioned Egocentric Prediction
Predicting how a person's first-person view will evolve (what action will follow, what plan completes a task, whether an in-progress shot will score) is fundamentally under-specified: the same context admits many plausib…
Pose EstimationNudging: Inference-time Alignment of LLMs via Guided Decoding
Large language models (LLMs) require alignment to effectively and safely follow user instructions. This process necessitates training an aligned version for every base model, resulting in significant computational overhe…
General KnowledgeGSM8KInstruction Followingmodel