Papers Video Alignment
“Video Alignment” 태그가 달린 논문 120편 · 필터 해제
EvalCrafter: Benchmarking and Evaluating Large Video Generation Models
The vision and language generative models have been overgrown in recent years. For video generation, various open-sourced models and public-available services have been developed to generate high-quality videos. However,…
BenchmarkingLanguage ModellingLarge Language ModelText-to-Video Generation+2STELLA: Continual Audio-Video Pre-training with Spatio-Temporal Localized Alignment
Continuously learning a variety of audio-video semantics over time is crucial for audio-related reasoning tasks in our ever-evolving world. However, this is a nontrivial problem and poses two critical challenges: sparse …
Continual LearningRepresentation LearningVideo AlignmentShow-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation
Significant advancements have been achieved in the realm of large-scale pre-trained text-to-video Diffusion Models (VDMs). However, previous methods either rely solely on pixel-based VDMs, which come with high computatio…
GPUText-to-Video GenerationVideo AlignmentVideo GenerationAudio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignment
Text-to-video retrieval systems have recently made significant progress by utilizing pre-trained models trained on large-scale image-text pairs. However, most of the latest methods primarily focus on the video modality w…
RetrievalText to Video RetrievalVideo AlignmentVideo RetrievalA Solution to CVPR'2023 AQTC Challenge: Video Alignment for Multi-Step Inference
Affordance-centric Question-driven Task Completion (AQTC) for Egocentric Assistant introduces a groundbreaking scenario. In this scenario, through learning instructional videos, AI assistants provide users with step-by-s…
Video AlignmentContentCTR: Frame-level Live Streaming Click-Through Rate Prediction with Multimodal Transformer
In recent years, live streaming platforms have gained immense popularity as they allow users to broadcast their videos and interact in real-time with hosts and peers. Due to the dynamic changes of live content, accurate …
Click-Through Rate PredictionDynamic Time WarpingVideo AlignmentSeeing the Pose in the Pixels: Learning Pose-Aware Representations in Vision Transformers
Human perception of surroundings is often guided by the various poses present within the environment. Many computer vision tasks, such as human action recognition and robot imitation learning, rely on pose-based entities…
Action ClassificationAction RecognitionImitation LearningPose Prediction+2Learning to Ground Instructional Articles in Videos through Narrations
In this paper we present an approach for localizing steps of procedural activities in narrated how-to videos. To deal with the scarcity of labeled data at scale, we source the step descriptions from a language knowledge …
ArticlesVideo AlignmentLearning by Aligning 2D Skeleton Sequences and Multi-Modality Fusion
This paper presents a self-supervised temporal video alignment framework which is useful for several fine-grained human activity understanding applications. In contrast with the state-of-the-art method of CASA, where seq…
RetrievalSelf-Supervised LearningVideo AlignmentSwap Attention in Spatiotemporal Diffusions for Text-to-Video Generation
With the explosive popularity of AI-generated content (AIGC), video generation has recently received a lot of attention. Generating videos guided by text instructions poses significant challenges, such as modeling the co…
Image GenerationText to Image GenerationText-to-Image GenerationText-to-Video Generation+2Edit As You Wish: Video Caption Editing with Multi-grained User Control
Automatically narrating videos in natural language complying with user requests, i.e. Controllable Video Captioning task, can help people manage massive videos with desired intentions. However, existing works suffer from…
AttributePositionTripletVideo Alignment+2Video alignment using unsupervised learning of local and global features
In this paper, we tackle the problem of video alignment, the process of matching the frames of a pair of videos containing similar actions. The main challenge in video alignment is that accurate correspondence should be …
Dynamic Time WarpingHuman DetectionPose EstimationTime Series+2Zero-Shot Video Editing Using Off-The-Shelf Image Diffusion Models
Large-scale text-to-image diffusion models achieve unprecedented success in image generation and editing. However, how to extend such success to video editing is unclear. Recent initial attempts at video editing require …
Video AlignmentVideo EditingAligning Step-by-Step Instructional Diagrams to Video Demonstrations
Multimodal alignment facilitates the retrieval of instances from one modality when queried using another. In this paper, we consider a novel setting where such an alignment is between (i) instruction steps that are depic…
Contrastive LearningImage RetrievalVideo AlignmentVideo RetrievalVADER: Video Alignment Differencing and Retrieval
We propose VADER, a spatio-temporal matching, alignment, and change summarization method to help fight misinformation spread via manipulated videos. VADER matches and coarsely aligns partial video fragments to candidate …
MisinformationRetrievalTemporal LocalizationVideo AlignmentWeakly Supervised Video Representation Learning with Unaligned Text for Sequential Videos
Sequential video understanding, as an emerging video understanding task, has driven lots of researchers' attention because of its goal-oriented nature. This paper studies weakly supervised sequential video understanding …
Representation LearningSentenceVideo AlignmentVideo UnderstandingWeakly-supervised Representation Learning for Video Alignment and Analysis
Many tasks in video analysis and understanding boil down to the need for frame-based feature learning, aiming to encapsulate the relevant visual content so as to enable simpler and easier subsequent processing. While sup…
Representation LearningVideo AlignmentPIDRo: Parallel Isomeric Attention with Dynamic Routing for Text-Video Retrieval
Text-video retrieval is a fundamental task with high practical value in multi-modal research. Inspired by the great success of pre-trained image-text models with large-scale data, such as CLIP, many methods are propo…
Representation LearningRetrievalSentenceVideo Alignment+2Self-supervised and Weakly Supervised Contrastive Learning for Frame-wise Action Representations
Previous work on action representation learning focused on global representations for short video clips. In contrast, many practical applications, such as video alignment, strongly demand learning the intensive represent…
Action ClassificationContrastive LearningRepresentation LearningRetrieval+2Learning a Grammar Inducer from Massive Uncurated Instructional Videos
Video-aided grammar induction aims to leverage video information for finding more accurate syntactic grammars for accompanying text. While previous work focuses on building systems for inducing grammars on text that are …
Language AcquisitionVideo Alignment