ConVRT: Consistent Video Restoration Through Turbulence with Test-time Optimization of Neural Video Representations
tmospheric turbulence presents a significant challenge in long-range imaging. Current restoration algorithms often struggle with temporal inconsistency, as well as limited generalization ability across varying turbulence levels and scene content different than the training data. To tackle these issues, we introduce a self-supervised method, Consistent Video Restoration through Turbulence (ConVRT) a test-time optimization method featuring a neural video representation designed to enhance temporal consistency in restoration. A key innovation of ConVRT is the integration of a pretrained vision-language model (CLIP) for semantic-oriented supervision, which steers the restoration towards sharp, photorealistic images in the CLIP latent space. We further develop a principled selection strategy of text prompts, based on their statistical correlation with a perceptual metric. ConVRT's test-time optimization allows it to adapt to a wide range of real-world turbulence conditions, effectively leveraging the insights gained from pre-trained models on simulated data. ConVRT offers a comprehensive and effective solution for mitigating real-world turbulence in dynamic videos.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingVideo RestorationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PMR: Physical Model-Driven Multi-Stage Restoration of Turbulent Dynamic Videos
Geometric distortions and blurring caused by atmospheric turbulence degrade the quality of long-range dynamic scene videos. Existing methods struggle with restoring edge details and eliminating mixed distortions, especia…
Motion SegmentationVideo RestorationTurb-Seg-Res: A Segment-then-Restore Pipeline for Dynamic Videos with Atmospheric Turbulence
Tackling image degradation due to atmospheric turbulence, particularly in dynamic environment, remains a challenge for long-range imaging systems. Existing techniques have been primarily designed for static scenes or sce…
Motion SegmentationOptical Flow EstimationSegmentationVideo RestorationSubsampled Turbulence Removal Network
We present a deep-learning approach to restore a sequence of turbulence-distorted video frames from turbulent deformations and space-time varying blurs. Instead of requiring a massive training sample size in deep network…
Data AugmentationRMFAT: Recurrent Multi-scale Feature Atmospheric Turbulence Mitigator
Atmospheric turbulence severely degrades video quality by introducing distortions such as geometric warping, blur, and temporal flickering, posing significant challenges to both visual clarity and temporal consistency. C…
Video RestorationPhysical prior guided cooperative learning framework for joint turbulence degradation estimation and infrared video restoration
Infrared imaging and turbulence strength measurements are in widespread demand in many fields. This paper introduces a Physical Prior Guided Cooperative Learning (P2GCL) framework to jointly enhance atmospheric turbulenc…
Image RestorationVideo Restoration