Space-Time-Aware Multi-Resolution Video Enhancement
We consider the problem of space-time super-resolution (ST-SR): increasing spatial resolution of video frames and simultaneously interpolating frames to increase the frame rate. Modern approaches handle these axes one at a time. In contrast, our proposed model called STARnet super-resolves jointly in space and time. This allows us to leverage mutually informative relationships between time and space: higher resolution can provide more detailed information about motion, and higher frame-rate can provide better pixel alignment. The components of our model that generate latent low- and high-resolution representations during ST-SR can be used to finetune a specialized mechanism for just spatial or just temporal super-resolution. Experimental results demonstrate that STARnet improves the performances of space-time, spatial, and temporal video super-resolution by substantial margins on publicly available datasets.
Code (1)
Tasks
Super-ResolutionVideo EnhancementVideo Super-ResolutionSimilar Papers 제목 키워드 기반
VEnhancer: Generative Space-Time Enhancement for Video Generation
We present VEnhancer, a generative space-time enhancement framework that improves the existing text-to-video results by adding more details in spatial domain and synthetic detailed motion in temporal domain. Given a gene…
Data AugmentationSuper-ResolutionVideo GenerationVideo Super-ResolutionDiffST: Spatiotemporal-Aware Diffusion for Real-World Space-Time Video Super-Resolution
Diffusion-based models have shown strong performance in video super-resolution (VSR) and video frame interpolation (VFI). However, their role in the coupled space-time video super-resolution (STVSR) setting remains limit…
Space-time Video Super-resolutionVideo Frame InterpolationMoTIF: Learning Motion Trajectories with Local Implicit Neural Functions for Continuous Space-Time Video Super-Resolution
This work addresses continuous space-time video super-resolution (C-STVSR) that aims to up-scale an input video both spatially and temporally by any scaling factors. One key challenge of C-STVSR is to propagate informati…
Motion InterpolationSpace-time Video Super-resolutionSuper-ResolutionVideo Super-ResolutionLDDR: Linear-DPP-Based Dynamic-Resolution Frame Sampling for Video MLLMs
Video understanding in multimodal large language models requires selecting informative frames from long, redundant videos under limited visual-token budgets. Existing methods often rely on uniform sampling, point-wise re…
Dynamic Video Generation: Shaping Video Generation Across Time and Space
Diffusion models have achieved impressive performance in video generation, but their iterative denoising process remains computationally expensive due to the large number of tokens processed at each timestep. Recently, p…
Video Generation