paper-with-me

홈 › Papers

Real-Time Video Super-Resolution by Joint Local Inference and Global Parameter Estimation

2021-05-06 · Noam Elron, Alex Itskovich, Shahar S. Yuval, Noam Levy

The state of the art in video super-resolution (SR) are techniques based on deep learning, but they perform poorly on real-world videos (see Figure 1). The reason is that training image-pairs are commonly created by downscaling a high-resolution image to produce a low-resolution counterpart. Deep models are therefore trained to undo downscaling and do not generalize to super-resolving real-world images. Several recent publications present techniques for improving the generalization of learning-based SR, but are all ill-suited for real-time application. We present a novel approach to synthesizing training data by simulating two digital-camera image-capture processes at different scales. Our method produces image-pairs in which both images have properties of natural images. Training an SR model using this data leads to far better generalization to real-world images and videos. In addition, deep video-SR models are characterized by a high operations-per-pixel count, which prohibits their application in real-time. We present an efficient CNN architecture, which enables real-time application of video SR on low-power edge-devices. We split the SR task into two sub-tasks: a control-flow which estimates global properties of the input video and adapts the weights and biases of a processing-CNN that performs the actual processing. Since the process-CNN is tailored to the statistics of the input, its capacity kept low, while retaining effectivity. Also, since video-statistics evolve slowly, the control-flow operates at a much lower rate than the video frame-rate. This reduces the overall computational load by as much as two orders of magnitude. This framework of decoupling the adaptivity of the algorithm from the pixel processing, can be applied in a large family of real-time video enhancement applications, e.g., video denoising, local tone-mapping, stabilization, etc.

📄 PDF Abstract BibTeX arXiv:2105.02794

Code (0)

등록된 구현이 없습니다.

Tasks

Denoisingparameter estimationSuper-ResolutionTone MappingVideo DenoisingVideo EnhancementVideo Super-Resolution

Similar Papers 제목 키워드 기반

Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation

2016-11-16 · CVPR 2017 7 · Jose Caballero, Christian Ledig, Andrew Aitken, Alejandro Acosta 외

Convolutional neural networks have enabled accurate image super-resolution in real-time. However, recent attempts to benefit from temporal correlations in video super-resolution have been limited to naive or inefficient …

Motion CompensationVideo Super-Resolution

Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions

2026-06-08 · Luxury, Jie Huang, Zihao Fan, Xiaoxiao Ma 외 arxiv

While recent autoregressive video diffusion models achieve remarkable streaming quality, they remain confined to low resolutions (e.g., 480P), leaving efficient, scalable, real-time high-resolution video generation a fun…

Video Generation

Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution

2022-08-05 · Zhongwei Qiu, Huan Yang, Jianlong Fu, Dongmei Fu

Compressed video super-resolution (VSR) aims to restore high-resolution frames from compressed low-resolution counterparts. Most recent VSR approaches often enhance an input frame by borrowing relevant textures from neig…

Super-ResolutionVideo EnhancementVideo Super-Resolution

3DAttGAN: A 3D Attention-based Generative Adversarial Network for Joint Space-Time Video Super-Resolution

2024-07-24 · Congrui Fu, Hui Yuan, Liquan Shen, Raouf Hamzaoui 외

In many applications, including surveillance, entertainment, and restoration, there is a need to increase both the spatial resolution and the frame rate of a video sequence. The aim is to improve visual quality, refine d…

Generative Adversarial NetworkSpace-time Video Super-resolutionSuper-ResolutionVideo Super-Resolution

Photorealistic Video Generation with Diffusion Models

2023-12-11 · Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu 외

We present W.A.L.T, a transformer-based approach for photorealistic video generation via diffusion modeling. Our approach has two key design decisions. First, we use a causal encoder to jointly compress images and videos…

Super-ResolutionText-to-Video GenerationVideo GenerationVideo Super-Resolution