Papers Video Alignment
“Video Alignment” 태그가 달린 논문 120편 · 필터 해제
LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretation
Recent advancements in large multimodal models (LMMs) have driven substantial progress in both text-to-video (T2V) generation and video-to-text (V2T) interpretation tasks. However, current AI-generated videos (AIGVs) sti…
BenchmarkingQuestion AnsweringText-to-Video GenerationVideo Alignment+1DAPE: Dual-Stage Parameter-Efficient Fine-Tuning for Consistent Video Editing with Diffusion Models
Video generation based on diffusion models presents a challenging multimodal task, with video editing emerging as a pivotal direction in this field. Recent video editing approaches primarily fall into two categories: tra…
parameter-efficient fine-tuningVideo AlignmentVideo EditingVideo GenerationHunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation
Customized video generation aims to produce videos featuring specific subjects under flexible user-defined conditions, yet existing methods often struggle with identity consistency and limited input modalities. In this p…
Human-Domain Subject-to-VideoSingle-Domain Subject-to-VideoVideo AlignmentVideo GenerationDyST-XL: Dynamic Layout Planning and Content Control for Compositional Text-to-Video Generation
Compositional text-to-video generation, which requires synthesizing dynamic scenes with multiple interacting entities and precise spatial-temporal relationships, remains a critical challenge for diffusion-based models. E…
AttributeDenoisingText-to-Video GenerationVideo Alignment+1Video4DGen: Enhancing Video and 4D Generation through Mutual Optimization
The advancement of 4D (i.e., sequential 3D) generation opens up new possibilities for lifelike experiences in various applications, where users can explore dynamic objects or characters from any viewpoint. Meanwhile, vid…
3D GenerationVideo AlignmentVideo GenerationVRMDiff: Text-Guided Video Referring Matting Generation of Diffusion
We propose a new task, video referring matting, which obtains the alpha matte of a specified instance by inputting a referring caption. We treat the dense prediction task of matting as video generation, leveraging the te…
Image MattingVideo AlignmentVideo GenerationDeep Understanding of Sign Language for Sign to Subtitle Alignment
The objective of this work is to align asynchronous subtitles in sign language videos with limited labelled data. To achieve this goal, we propose a novel framework with the following contributions: (1) we leverage funda…
TranslationVideo AlignmentInference-Time Text-to-Video Alignment with Diffusion Latent Beam Search
The remarkable progress in text-to-video diffusion models enables photorealistic generations, although the contents of the generated video often include unnatural movement or deformation, reverse playback, and motionless…
DenoisingVideo AlignmentVideo GenerationSound Bridge: Associating Egocentric and Exocentric Videos via Audio Cues
Understanding human behavior and the environmental information in the egocentric video is very challenging due to the invisibility of some actions (e.g., laughing and sneezing) and the local nature of the first-perso…
Action RecognitionScene RecognitionVideo AlignmentSmooth-Foley: Creating Continuous Sound for Video-to-Audio Generation Under Semantic Guidance
The video-to-audio (V2A) generation task has drawn attention in the field of multimedia due to the practicality in producing Foley sound. Semantic and temporal conditions are fed to the generation model to indicate sound…
Audio GenerationVideo AlignmentImproving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback
Large text-to-video models hold immense potential for a wide range of downstream applications. However, these models struggle to accurately depict dynamic object interactions, often resulting in unrealistic movements and…
ObjectOffline RLOptical Flow EstimationText-to-Video Generation+2HunyuanVideo: A Systematic Framework For Large Video Generative Models
Recent advancements in video generation have significantly impacted daily life for both individuals and industries. However, the leading video generation models remain closed-source, resulting in a notable performance ga…
Video AlignmentVideo GenerationVideoRepair: Improving Text-to-Video Generation via Misalignment Evaluation and Localized Refinement
Recent text-to-video (T2V) diffusion models have demonstrated impressive generation capabilities across various domains. However, these models often generate videos that have misalignments with text prompts, especially w…
Text-to-Video GenerationVideo AlignmentVideo GenerationNeuro-Symbolic Evaluation of Text-to-Video Models using Formal Verification
Recent advancements in text-to-video models such as Sora, Gen-3, MovieGen, and CogVideoX are pushing the boundaries of synthetic video generation, with adoption seen in fields like robotics, autonomous driving, and enter…
Autonomous DrivingText-to-Video GenerationVideo AlignmentVideo GenerationKoala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content
As visual generation technologies continue to advance, the scale of video datasets has expanded rapidly, and the quality of these datasets is critical to the performance of video generation models. We argue that temporal…
Video AlignmentVideo GenerationT2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Design
In this paper, we focus on enhancing a diffusion-based text-to-video (T2V) model during the post-training phase by distilling a highly capable consistency model from a pretrained T2V model. Our proposed method, T2V-Turbo…
Video AlignmentVideo GenerationLearning to Localize Actions in Instructional Videos with LLM-Based Multi-Pathway Text-Video Alignment
Learning to localize temporal boundaries of procedure steps in instructional videos is challenging due to the limited availability of annotated large-scale training videos. Recent works focus on learning the cross-modal …
Contrastive Learningcross-modal alignmentSemantic SimilaritySemantic Textual Similarity+2Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations
Emotion Recognition in Conversations (ERCs) is a vital area within multimodal interaction research, dedicated to accurately identifying and classifying the emotions expressed by speakers throughout a conversation. Tradit…
Emotion RecognitionMambamultimodal interactionVideo AlignmentSelf-Supervised Contrastive Learning for Videos using Differentiable Local Alignment
Robust frame-wise embeddings are essential to perform video analysis and understanding tasks. We present a self-supervised method for representation learning based on aligning temporal video sequences. Our framework uses…
Action RecognitionContrastive LearningRepresentation LearningVideo AlignmentSync from the Sea: Retrieving Alignable Videos from Large-Scale Datasets
Temporal video alignment aims to synchronize the key events like object interactions or action phase transitions in two videos. Such methods could benefit various video editing, processing, and understanding tasks. Howev…
Video AlignmentVideo EditingVideo Retrieval