Improving Video Generation for Multi-functional Applications
In this paper, we aim to improve the state-of-the-art video generative adversarial networks (GANs) with a view towards multi-functional applications. Our improved video GAN model does not separate foreground from background nor dynamic from static patterns, but learns to generate the entire video clip conjointly. Our model can thus be trained to generate - and learn from - a broad set of videos with no restriction. This is achieved by designing a robust one-stream video generation architecture with an extension of the state-of-the-art Wasserstein GAN framework that allows for better convergence. The experimental results show that our improved video GAN model outperforms state-of-theart video generative models on multiple challenging datasets. Furthermore, we demonstrate the superiority of our model by successfully extending it to three challenging problems: video colorization, video inpainting, and future prediction. To the best of our knowledge, this is the first work using GANs to colorize and inpaint video clips.
Code (1)
Tasks
ColorizationFuture predictionVideo GenerationVideo InpaintingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
OmniVDiff: Omni Controllable Video Diffusion for Generation and Understanding
In this paper, we propose a novel framework for controllable video diffusion, OmniVDiff, aiming to synthesize and comprehend multiple video visual content in a single diffusion model. To achieve this, OmniVDiff treats al…
Semantic SegmentationVideo GenerationVideo UnderstandingKandinsky 3: Text-to-Image Synthesis for Multifunctional Generative Framework
Text-to-image (T2I) diffusion models are popular for introducing image manipulation methods, such as editing, image fusion, inpainting, etc. At the same time, image-to-video (I2V) and text-to-video (T2V) models are also …
Image GenerationImage ManipulationContinuous-Time Video Generation via Learning Motion Dynamics with Neural ODE
In order to perform unconditional video generation, we must learn the distribution of the real-world videos. In an effort to synthesize high-quality videos, various studies attempted to learn a mapping function between n…
Unconditional Video GenerationVideo GenerationOpenVVC: a Lightweight Software Decoder for the Versatile Video Coding Standard
In the recent years, users requirements for higher resolution, coupled with the apparition of new multimedia applications, have created the need for a new video coding standard. The new generation video coding standard, …
DecoderVideo2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisit
UI videos provide a natural input for generating interactive webpages, as they capture both webpage appearance and action-triggered state transitions. However, directly applying video-capable vision-language models to th…
Code Generation