LAMV: Learning to Align and Match Videos With Kernelized Temporal Layers
This paper considers a learnable approach for comparing and aligning videos. Our architecture builds upon and revisits temporal match kernels within neural networks: we propose a new temporal layer that finds temporal alignments by maximizing the scores between two sequences of vectors, according to a time-sensitive similarity metric parametrized in the Fourier domain. We learn this layer with a temporal proposal strategy, in which we minimize a triplet loss that takes into account both the localization accuracy and the recognition rate. We evaluate our approach on video alignment, copy detection and event retrieval. Our approach outperforms the state on the art on temporal video alignment and video copy detection datasets in comparable setups. It also attains the best reported results for particular event search, while precisely aligning videos.
Code (1)
Tasks
Copy DetectionRetrievalTripletVideo AlignmentVideo RetrievalSimilar Papers 제목 키워드 기반
Circulant temporal encoding for video retrieval and temporal alignment
We address the problem of specific video event retrieval. Given a query video of a specific event, e.g., a concert of Madonna, the goal is to retrieve other videos of the same event that temporally overlap with the query…
RetrievalVideo RetrievalVADER: Video Alignment Differencing and Retrieval
We propose VADER, a spatio-temporal matching, alignment, and change summarization method to help fight misinformation spread via manipulated videos. VADER matches and coarsely aligns partial video fragments to candidate …
MisinformationRetrievalTemporal LocalizationVideo AlignmentLAMVI-2: A Visual Tool for Comparing and Tuning Word Embedding Models
Tuning machine learning models, particularly deep learning architectures, is a complex process. Automated hyperparameter tuning algorithms often depend on specific optimization metrics. However, in many situations, a dev…
Deep LearningHyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition
Recent attempts mainly focus on learning deep representations for each video individually under the episodic meta-learning regime and then performing temporal alignment to match query and support videos. However, they st…
Action RecognitionFew-Shot action recognitionFew Shot Action RecognitionMeta-Learning+2Temporal Cycle-Consistency Learning
We introduce a self-supervised representation learning method based on the task of temporal alignment between videos. The method trains a network using temporal cycle consistency (TCC), a differentiable cycle-consistency…
Anomaly DetectionRepresentation LearningSelf-Supervised LearningVideo Alignment