Prompt-based Consistent Video Colorization
Existing video colorization methods struggle with temporal flickering or demand extensive manual input. We propose a novel approach automating high-fidelity video colorization using rich semantic guidance derived from language and segmentation. We employ a language-conditioned diffusion model to colorize grayscale frames. Guidance is provided via automatically generated object masks and textual prompts; our primary automatic method uses a generic prompt, achieving state-of-the-art results without specific color input. Temporal stability is achieved by warping color information from previous frames using optical flow (RAFT); a correction step detects and fixes inconsistencies introduced by warping. Evaluations on standard benchmarks (DAVIS30, VIDEVO20) show our method achieves state-of-the-art performance in colorization accuracy (PSNR) and visual realism (Colorfulness, CDC), demonstrating the efficacy of automated prompt-based guidance for consistent video colorization.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning
Video colorization is a challenging and highly ill-posed problem. Although recent years have witnessed remarkable progress in single image colorization, there is relatively less research effort on video colorization and …
ColorizationImage ColorizationAutomatic Temporally Coherent Video Colorization
Greyscale image colorization for applications in image restoration has seen significant improvements in recent years. Many of these techniques that use learning-based methods struggle to effectively colorize sparse input…
ColorizationImage ColorizationImage RestorationImage-to-Image Translation+1L-C4: Language-Based Video Colorization for Creative and Consistent Color
Automatic video colorization is inherently an ill-posed problem because each monochrome frame has multiple optional color candidates. Previous exemplar-based video colorization methods restrict the user's imagination due…
ColorizationImage ColorizationVCGAN: Video Colorization with Hybrid Generative Adversarial Network
We propose a hybrid recurrent Video Colorization with Hybrid Generative Adversarial Network (VCGAN), an improved approach to video colorization using end-to-end learning. The VCGAN addresses two prevalent issues in the v…
ColorizationGenerative Adversarial NetworkImage ColorizationFlowChroma -- A Deep Recurrent Neural Network for Video Colorization
We develop an automated video colorization framework that minimizes the flickering of colors across frames. If we apply image colorization techniques to successive frames of a video, they treat each frame as a separate c…
ColorizationDecoderImage Colorization