VORNet: Spatio-temporally Consistent Video Inpainting for Object Removal
Video object removal is a challenging task in video processing that often requires massive human efforts. Given the mask of the foreground object in each frame, the goal is to complete (inpaint) the object region and generate a video without the target object. While recently deep learning based methods have achieved great success on the image inpainting task, they often lead to inconsistent results between frames when applied to videos. In this work, we propose a novel learning-based Video Object Removal Network (VORNet) to solve the video object removal task in a spatio-temporally consistent manner, by combining the optical flow warping and image-based inpainting model. Experiments are done on our Synthesized Video Object Removal (SVOR) dataset based on the YouTube-VOS video segmentation dataset, and both the objective and subjective evaluation demonstrate that our VORNet generates more spatially and temporally consistent videos compared with existing methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Image InpaintingObjectOne-shot visual object segmentationOptical Flow EstimationVideo InpaintingVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Spatiotemporally Consistent Indoor Lighting Estimation with Diffusion Priors
Indoor lighting estimation from a single image or video remains a challenge due to its highly ill-posed nature, especially when the lighting condition of the scene varies spatially and temporally. We propose a method tha…
Zero-shot GeneralizationDeep Video Inpainting
Video inpainting aims to fill spatio-temporal holes with plausible content in a video. Despite tremendous progress of deep neural networks for image inpainting, it is challenging to extend these methods to the video doma…
DecoderImage InpaintingOptical Flow EstimationVideo Denoising+2Occlusion-Aware Video Object Inpainting
Conventional video inpainting is neither object-oriented nor occlusion-aware, making it liable to obvious artifacts when large occluded object regions are inpainted. This paper presents occlusion-aware video object inpai…
ObjectTexture SynthesisVideo InpaintingMotionAura: Generating High-Quality and Motion Consistent Videos using Discrete Diffusion
The spatio-temporal complexity of video data presents significant challenges in tasks such as compression, generation, and inpainting. We present four key contributions to address the challenges of spatiotemporal video p…
Denoisingparameter-efficient fine-tuningQuantizationText-to-Video Generation+3Decoupled Spatial-Temporal Transformer for Video Inpainting
Video inpainting aims to fill the given spatiotemporal holes with realistic appearance but is still a challenging task even with prosperous deep learning approaches. Recent works introduce the promising Transformer archi…
Video Inpainting