Contrastive Sequential-Diffusion Learning: Non-linear and Multi-Scene Instructional Video Synthesis
Generated video scenes for action-centric sequence descriptions, such as recipe instructions and do-it-yourself projects, often include non-linear patterns, where the next video may need to be visually consistent not with the immediately preceding video but with earlier ones. Current multi-scene video synthesis approaches fail to meet these consistency requirements. To address this, we propose a contrastive sequential video diffusion method that selects the most suitable previously generated scene to guide and condition the denoising process of the next scene. The result is a multi-scene video that is grounded in the scene descriptions and coherent w.r.t. the scenes that require visual consistency. Experiments with action-centered data from the real world demonstrate the practicality and improved consistency of our model compared to previous work.
Code (1)
Tasks
DenoisingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
R3CD: Scene Graph to Image Generation with Relation-aware Compositional Contrastive Control Diffusion
Image generation tasks have achieved remarkable performance using large-scale diffusion models. However, these models are limited to capturing the abstract relations (viz., interactions excluding positional relations) am…
DenoisingImage GenerationRelationScene Graph Generation+1Similarity-Guided Diffusion for Contrastive Sequential Recommendation
In sequential recommendation systems, data augmentation and contrastive learning techniques have recently been introduced using diffusion models to achieve robust representation learning. However, most of the existing ap…
Contrastive LearningData AugmentationDenoisingRecommendation Systems+2Intent-aware Diffusion with Contrastive Learning for Sequential Recommendation
Contrastive learning has proven effective in training sequential recommendation models by incorporating self-supervised signals from augmented views. Most existing methods generate multiple views from the same interactio…
Contrastive LearningData AugmentationSequential RecommendationDiffusion-based Contrastive Learning for Sequential Recommendation
Contrastive learning has been effectively utilized to enhance the training of sequential recommendation models by leveraging informative self-supervised signals. Most existing approaches generate augmented views of the s…
Contrastive LearningSequential RecommendationImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis
Autoregressive models and their sequential factorization of the data likelihood have recently demonstrated great potential for image representation and synthesis. Nevertheless, they incorporate image context in a linear …
Image GenerationImage InpaintingText-to-Image Generation