Internal Video Inpainting by Implicit Long-range Propagation
We propose a novel framework for video inpainting by adopting an internal learning strategy. Unlike previous methods that use optical flow for cross-frame context propagation to inpaint unknown regions, we show that this can be achieved implicitly by fitting a convolutional neural network to known regions. Moreover, to handle challenging sequences with ambiguous backgrounds or long-term occlusion, we design two regularization terms to preserve high-frequency details and long-term temporal consistency. Extensive experiments on the DAVIS dataset demonstrate that the proposed method achieves state-of-the-art inpainting quality quantitatively and qualitatively. We further extend the proposed method to another challenging task: learning to remove an object from a video giving a single object mask in only one frame in a 4K video.
Code (1)
Tasks
4kObjectOptical Flow EstimationVideo InpaintingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
An Internal Learning Approach to Video Inpainting
We propose a novel video inpainting algorithm that simultaneously hallucinates missing appearance and motion (optical flow) information, building upon the recent 'Deep Image Prior' (DIP) that exploits convolutional netwo…
Optical Flow EstimationVideo InpaintingUnified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising
Generating long videos remains a fundamental challenge, and achieving high controllability in video inpainting and outpainting is particularly demanding. To address both of these challenges simultaneously and achieve con…
Video GenerationVideo InpaintingHNeRV: A Hybrid Neural Representation for Videos
Implicit neural representations store videos as neural networks and have performed well for various vision tasks such as video compression and denoising. With frame index or positional index as input, implicit representa…
DecoderDenoisingregressionVideo Compression+2Attentive Normalization for Conditional Image Generation
Traditional convolution-based generative adversarial networks synthesize images based on hierarchical local operations, where long-range dependency relation is implicitly modeled with a Markov chain. It is still not suff…
Conditional Image GenerationImage GenerationSemantic correspondenceSemantic Similarity+1Infusion: internal diffusion for inpainting of dynamic textures and complex motion
Video inpainting is the task of filling a region in a video in a visually convincing manner. It is very challenging due to the high dimensionality of the data and the temporal consistency required for obtaining convincin…
Image InpaintingOptical Flow EstimationVideo Inpainting