Video Frame Interpolation via Generalized Deformable Convolution
Video frame interpolation aims at synthesizing intermediate frames from nearby source frames while maintaining spatial and temporal consistencies. The existing deep-learning-based video frame interpolation methods can be roughly divided into two categories: flow-based methods and kernel-based methods. The performance of flow-based methods is often jeopardized by the inaccuracy of flow map estimation due to oversimplified motion models, while that of kernel-based methods tends to be constrained by the rigidity of kernel shape. To address these performance-limiting issues, a novel mechanism named generalized deformable convolution is proposed, which can effectively learn motion information in a data-driven manner and freely select sampling points in space-time. We further develop a new video frame interpolation method based on this mechanism. Our extensive experiments demonstrate that the new method performs favorably against the state-of-the-art, especially when dealing with complex motions.
Code (1)
Tasks
Video Frame InterpolationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Enhancing Deformable Convolution based Video Frame Interpolation with Coarse-to-fine 3D CNN
This paper presents a new deformable convolution-based video frame interpolation (VFI) method, using a coarse to fine 3D CNN to enhance the multi-flow prediction. This model first extracts spatio-temporal features at mul…
Video Frame InterpolationConvolutional Neural Networks vs. Deformable Image Registration For Medical Slice Interpolation
Medical image slice interpolation is an active field of research. The methods for this task can be categorized into two broad groups: intensity-based and object-based interpolation methods. While intensity-based methods …
Image RegistrationObjectVideo Frame InterpolationVideo Frame Interpolation Based on Deformable Kernel Region
Video frame interpolation task has recently become more and more prevalent in the computer vision field. At present, a number of researches based on deep learning have achieved great success. Most of them are either base…
Optical Flow EstimationVideo Frame InterpolationE-VFIA : Event-Based Video Frame Interpolation with Attention
Video frame interpolation (VFI) is a fundamental vision task that aims to synthesize several frames between two consecutive original video images. Most algorithms aim to accomplish VFI by using only keyframes, which is a…
Optical Flow EstimationVideo Frame InterpolationTemporal Modulation Network for Controllable Space-Time Video Super-Resolution
Space-time video super-resolution (STVSR) aims to increase the spatial and temporal resolutions of low-resolution and low-frame-rate videos. Recently, deformable convolution based methods have achieved promising STVSR pe…
Space-time Video Super-resolutionSuper-ResolutionVideo Super-Resolution