Deformable 3D Convolution for Video Super-Resolution
The spatio-temporal information among video sequences is significant for video super-resolution (SR). However, the spatio-temporal information cannot be fully used by existing video SR methods since spatial feature extraction and temporal motion compensation are usually performed sequentially. In this paper, we propose a deformable 3D convolution network (D3Dnet) to incorporate spatio-temporal information from both spatial and temporal dimensions for video SR. Specifically, we introduce deformable 3D convolution (D3D) to integrate deformable convolution with 3D convolution, obtaining both superior spatio-temporal modeling capability and motion-aware modeling flexibility. Extensive experiments have demonstrated the effectiveness of D3D in exploiting spatio-temporal information. Comparative results show that our network achieves state-of-the-art SR performance. Code is available at: https://github.com/XinyiYing/D3Dnet.
Code (1)
Tasks
Motion CompensationSuper-ResolutionVideo Super-ResolutionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Deformable Kernel Convolutional Network for Video Extreme Super-Resolution
Video super-resolution, which attempts to reconstruct high-resolution video frames from their corresponding low-resolution versions, has received increasingly more attention in recent years. Most existing approaches opt …
Super-ResolutionVideo Super-ResolutionUnderstanding Deformable Alignment in Video Super-Resolution
Deformable convolution, originally proposed for the adaptation to geometric variations of objects, has recently shown compelling performance in aligning multiple frames and is increasingly adopted for video super-resolut…
DiversityOptical Flow EstimationSuper-ResolutionVideo Super-ResolutionTemporal Modulation Network for Controllable Space-Time Video Super-Resolution
Space-time video super-resolution (STVSR) aims to increase the spatial and temporal resolutions of low-resolution and low-frame-rate videos. Recently, deformable convolution based methods have achieved promising STVSR pe…
Space-time Video Super-resolutionSuper-ResolutionVideo Super-ResolutionEDVR: Video Restoration with Enhanced Deformable Convolutional Networks
Video restoration tasks, including super-resolution, deblurring, etc, are drawing increasing attention in the computer vision community. A challenging benchmark named REDS is released in the NTIRE19 Challenge. This new b…
DeblurringSuper-ResolutionVideo EnhancementVideo Restoration+1Deformable Non-local Network for Video Super-Resolution
The video super-resolution (VSR) task aims to restore a high-resolution (HR) video frame by using its corresponding low-resolution (LR) frame and multiple neighboring frames. At present, many deep learning-based VSR meth…
Optical Flow EstimationSuper-ResolutionVideo Super-Resolution