Revisiting Temporal Modeling for Video Super-resolution
Video super-resolution plays an important role in surveillance video analysis and ultra-high-definition video display, which has drawn much attention in both the research and industrial communities. Although many deep learning-based VSR methods have been proposed, it is hard to directly compare these methods since the different loss functions and training datasets have a significant impact on the super-resolution results. In this work, we carefully study and compare three temporal modeling methods (2D CNN with early fusion, 3D CNN with slow fusion and Recurrent Neural Network) for video super-resolution. We also propose a novel Recurrent Residual Network (RRN) for efficient video super-resolution, where residual learning is utilized to stabilize the training of RNN and meanwhile to boost the super-resolution performance. Extensive experiments show that the proposed RRN is highly computational efficiency and produces temporal consistent VSR results with finer details than other temporal modeling methods. Besides, the proposed method achieves state-of-the-art results on several widely used benchmarks.
Code (2)
Tasks
Computational EfficiencySuper-ResolutionVideo Super-ResolutionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Revisiting Temporal Alignment for Video Restoration
Long-range temporal alignment is critical yet challenging for video restoration tasks. Recently, some works attempt to divide the long-range alignment into several sub-alignments and handle them progressively. Although t…
DeblurringDenoisingMotion CompensationSuper-Resolution+2Look Back and Forth: Video Super-Resolution with Explicit Temporal Difference Modeling
Temporal modeling is crucial for video super-resolution. Most of the video super-resolution methods adopt the optical flow or deformable convolution for explicitly motion compensation. However, such temporal modeling tec…
Motion CompensationOptical Flow EstimationSuper-ResolutionVideo Super-ResolutionDeformable 3D Convolution for Video Super-Resolution
The spatio-temporal information among video sequences is significant for video super-resolution (SR). However, the spatio-temporal information cannot be fully used by existing video SR methods since spatial feature extra…
Motion CompensationSuper-ResolutionVideo Super-ResolutionRevisiting Temporal Modeling for CLIP-based Image-to-Video Knowledge Transferring
Image-text pretrained models, e.g., CLIP, have shown impressive general multi-modal knowledge learned from large-scale image-text data pairs, thus attracting increasing attention for their potential to improve visual rep…
Representation LearningRetrievalText RetrievalVideo Recognition+2Revisiting Temporal Modeling for Video-based Person ReID
Video-based person reID is an important task, which has received much attention in recent years due to the increasing demand in surveillance and camera networks. A typical video-based person reID system consists of three…