Flow-Guided Video Inpainting with Scene Templates
We consider the problem of filling in missing spatio-temporal regions of a video. We provide a novel flow-based solution by introducing a generative model of images in relation to the scene (without missing regions) and mappings from the scene to images. We use the model to jointly infer the scene template, a 2D representation of the scene, and the mappings. This ensures consistency of the frame-to-frame flows generated to the underlying scene, reducing geometric distortions in flow based inpainting. The template is mapped to the missing regions in the video by a new L2-L1 interpolation scheme, creating crisp inpaintings and reducing common blur and distortion artifacts. We show on two benchmark datasets that our approach out-performs state-of-the-art quantitatively and in user studies.
Code (0)
등록된 구현이 없습니다.
Tasks
Video InpaintingSimilar Papers 제목 키워드 기반
Deep Flow-Guided Video Inpainting
Video inpainting, which aims at filling in missing regions of a video, remains challenging due to the difficulty of preserving the precise spatial and temporal coherence of video contents. In this work we propose a novel…
One-shot visual object segmentationOptical Flow EstimationVideo fixed region InpaintingVideo InpaintingFlow-Guided Diffusion for Video Inpainting
Video inpainting has been challenged by complex scenarios like large movements and low-light conditions. Current methods, including emerging diffusion models, face limitations in quality and efficiency. This paper introd…
DenoisingImage GenerationOptical Flow EstimationVideo InpaintingTowards An End-to-End Framework for Flow-Guided Video Inpainting
Optical flow, which captures motion information across frames, is exploited in recent video inpainting methods through propagating pixels along its trajectories. However, the hand-crafted flow-based processes in these me…
HallucinationOptical Flow EstimationSeeing Beyond the VisibleVideo InpaintingAdvanced Video Inpainting Using Optical Flow-Guided Efficient Diffusion
Recently, diffusion-based methods have achieved great improvements in the video inpainting task. However, these methods still face many challenges, such as maintaining temporal consistency and the time-consuming issue. T…
DenoisingOptical Flow EstimationVideo InpaintingError Compensation Framework for Flow-Guided Video Inpainting
The key to video inpainting is to use correlation information from as many reference frames as possible. Existing flow-based propagation methods split the video synthesis process into multiple steps: flow completion -> p…
Video Inpainting