PLOT-TAL -- Prompt Learning with Optimal Transport for Few-Shot Temporal Action Localization
This paper introduces a novel approach to temporal action localization (TAL) in few-shot learning. Our work addresses the inherent limitations of conventional single-prompt learning methods that often lead to overfitting due to the inability to generalize across varying contexts in real-world videos. Recognizing the diversity of camera views, backgrounds, and objects in videos, we propose a multi-prompt learning framework enhanced with optimal transport. This design allows the model to learn a set of diverse prompts for each action, capturing general characteristics more effectively and distributing the representation to mitigate the risk of overfitting. Furthermore, by employing optimal transport theory, we efficiently align these prompts with action features, optimizing for a comprehensive representation that adapts to the multifaceted nature of video data. Our experiments demonstrate significant improvements in action localization accuracy and robustness in few-shot settings on the standard challenging datasets of THUMOS-14 and EpicKitchens100, highlighting the efficacy of our multi-prompt optimal transport approach in overcoming the challenges of conventional few-shot TAL methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Action LocalizationDiversityFew-Shot LearningFew Shot Temporal Action LocalizationPrompt LearningTemporal Action LocalizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PLOT: Prompt Learning with Optimal Transport for Vision-Language Models
With the increasing attention to large vision-language models such as CLIP, there has been a significant amount of effort dedicated to building efficient prompts. Unlike conventional methods of only learning one single p…
Prompt LearningZegOT: Zero-shot Segmentation Through Optimal Transport of Text Prompts
Recent success of large-scale Contrastive Language-Image Pre-training (CLIP) has led to great promise in zero-shot semantic segmentation by transferring image-text aligned knowledge to pixel-level classification. However…
SegmentationSemantic SegmentationZero Shot SegmentationZero-Shot Semantic SegmentationPLOT: Enhancing Preference Learning via Optimal Transport
Preference learning in Large Language Models (LLMs) has advanced significantly, yet existing methods remain limited by modest performance gains, high computational costs, hyperparameter sensitivity, and insufficient mode…
MultistageOT: Multistage optimal transport infers trajectories from a snapshot of single-cell data
Single-cell RNA-sequencing captures a temporal slice, or a snapshot, of a cell differentiation process. A major bioinformatical challenge is the inference of differentiation trajectories from a single snapshot, and metho…
PLOT: Progressive Localization via Optimal Transport in Neural Causal Abstraction
Causal abstraction offers a principled framework for mechanistic interpretability, aligning a high-level causal model with the low-level computation realized by a neural network through counterfactual intervention analys…