Hierarchical Flow Diffusion for Efficient Frame Interpolation
Most recent diffusion-based methods still show a large gap compared to non-diffusion methods for video frame interpolation, in both accuracy and efficiency. Most of them formulate the problem as a denoising procedure in latent space directly, which is less effective caused by the large latent space. We propose to model bilateral optical flow explicitly by hierarchical diffusion models, which has much smaller search space in the denoising procedure. Based on the flow diffusion model, we then use a flow-guided image synthesizer to produce the final result. We train the flow diffusion model and the image synthesizer end to end. Our method achieves state of the art in accuracy, and 10+ times faster than other diffusion-based methods. The project page is at: https://hfd-interpolation.github.io.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingOptical Flow EstimationVideo Frame InterpolationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SNM-VFI: Symmetric Nonlinear Motion-Guided Generative Video Frame Interpolation
We propose Symmetric Nonlinear Motion-guided Generative Video Frame Interpolation (SNM-VFI), a training-free framework for motion-controllable generative video frame interpolation with pre-trained optical flow and video …
Video Frame InterpolationTANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation
We present TANGO, a framework for generating co-speech body-gesture videos. Given a few-minute, single-speaker reference video and target speech audio, TANGO produces high-fidelity videos with synchronized body gestures.…
cross-modal alignmentRetrievalvalidDepth-Aware Video Frame Interpolation
Video frame interpolation aims to synthesize nonexistent frames in-between the original frames. While significant advances have been made from the recent deep convolutional neural networks, the quality of interpolation i…
Optical Flow EstimationVideo Frame InterpolationEDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolation
Handling complex or nonlinear motion patterns has long posed challenges for video frame interpolation. Although recent advances in diffusion-based methods offer improvements over traditional optical flow-based approaches…
Optical Flow EstimationVideo Frame InterpolationRF-HiT: Rectified Flow Hierarchical Transformer for General Medical Image Segmentation
Accurate medical image segmentation requires both long-range contextual reasoning and precise boundary delineation, a task where existing transformer- and diffusion-based paradigms are frequently bottlenecked by quadrati…
Medical Image Segmentation