Preserving Semantic and Temporal Consistency for Unpaired Video-to-Video Translation
In this paper, we investigate the problem of unpaired video-to-video translation. Given a video in the source domain, we aim to learn the conditional distribution of the corresponding video in the target domain, without seeing any pairs of corresponding videos. While significant progress has been made in the unpaired translation of images, directly applying these methods to an input video leads to low visual quality due to the additional time dimension. In particular, previous methods suffer from semantic inconsistency (i.e., semantic label flipping) and temporal flickering artifacts. To alleviate these issues, we propose a new framework that is composed of carefully-designed generators and discriminators, coupled with two core objective functions: 1) content preserving loss and 2) temporal consistency loss. Extensive qualitative and quantitative evaluations demonstrate the superior performance of the proposed method against previous approaches. We further apply our framework to a domain adaptation task and achieve favorable results.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationTranslationSimilar Papers 제목 키워드 기반
Learning Temporally and Semantically Consistent Unpaired Video-to-video Translation Through Pseudo-Supervision From Synthetic Optical Flow
Unpaired video-to-video translation aims to translate videos between a source and a target domain without the need of paired training data, making it more feasible for real applications. Unfortunately, the translated vid…
Motion EstimationOptical Flow EstimationTranslationLong-Term Temporally Consistent Unpaired Video Translation from Simulated Surgical 3D Data
Research in unpaired video translation has mainly focused on short-term temporal consistency by conditioning on neighboring frames. However for transfer from simulated to photorealistic sequences, available information o…
Neural RenderingTranslationBeyond Consistency: Preserving Temporal Structure in Zero-Shot Video Editing
Existing zero-shot video editing methods rely on pre-trained diffusion models, successfully achieving spatial control and basic temporal consistency but fundamentally fail to preserve the video's original temporal struct…
Computational EfficiencyNOVA: Sparse Control, Dense Synthesis for Pair-Free Video Editing
Recent video editing models have achieved impressive results, but most still require large-scale paired datasets. Collecting such naturally aligned pairs at scale remains highly challenging and constitutes a critical bot…
Image EditingI2V-GAN: Unpaired Infrared-to-Visible Video Translation
Human vision is often adversely affected by complex environmental factors, especially in night vision scenarios. Thus, infrared cameras are often leveraged to help enhance the visual effects via detecting infrared radiat…
object-detectionObject DetectionTranslation