Optical Flow Regularization of Implicit Neural Representations for Video Frame Interpolation
Recent works have shown the ability of Implicit Neural Representations (INR) to carry meaningful representations of signal derivatives. In this work, we leverage this property to perform Video Frame Interpolation (VFI) by explicitly constraining the derivatives of the INR to satisfy the optical flow constraint equation. We achieve state of the art VFI on limited motion ranges using only a target video and its optical flow, without learning the interpolation operator from additional training data. We further show that constraining the INR derivatives not only allows to better interpolate intermediate frames but also improves the ability of narrow networks to fit the observed frames, which suggests potential applications to video compression and INR optimization.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationVideo CompressionVideo Frame InterpolationSimilar Papers 제목 키워드 기반
E-MoFlow: Learning Egomotion and Optical Flow from Event Data via Implicit Regularization
The estimation of optical flow and 6-DoF ego-motion, two fundamental tasks in 3D vision, has typically been addressed independently. For neuromorphic vision (e.g., event cameras), however, the lack of robust data associa…
Optical Flow EstimationDepth EstimationLearning Video Stabilization Using Optical Flow
We propose a novel neural network that infers the per-pixel warp fields for video stabilization from the optical flow fields of the input video. While previous learning based video stabilization methods attempt to implic…
Optical Flow EstimationVideo StabilizationFLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation
A majority of methods for video frame interpolation compute bidirectional optical flow between adjacent frames of a video, followed by a suitable warping algorithm to generate the output frames. However, approaches relyi…
Action RecognitionMotion MagnificationOptical Flow EstimationVideo Frame InterpolationInternal Video Inpainting by Implicit Long-range Propagation
We propose a novel framework for video inpainting by adopting an internal learning strategy. Unlike previous methods that use optical flow for cross-frame context propagation to inpaint unknown regions, we show that this…
4kObjectOptical Flow EstimationVideo InpaintingBi-FlowGS: Bridging Generative View Completion and Gaussian Geometry through Bidirectional Flow Co-Refinement
Sparse-view 3D scene reconstruction with 3D Gaussian Splatting (3DGS) is inherently underconstrained. Plausible renderings can also coexist with erroneous Gaussian geometry, as errors in positions or depths may be concea…
Video Restoration