Temporal Interpolation as an Unsupervised Pretraining Task for Optical Flow Estimation
The difficulty of annotating training data is a major obstacle to using CNNs for low-level tasks in video. Synthetic data often does not generalize to real videos, while unsupervised methods require heuristic losses. Proxy tasks can overcome these issues, and start by training a network for a task for which annotation is easier or which can be trained unsupervised. The trained network is then fine-tuned for the original task using small amounts of ground truth data. Here, we investigate frame interpolation as a proxy task for optical flow. Using real movies, we train a CNN unsupervised for temporal interpolation. Such a network implicitly estimates motion, but cannot handle untextured regions. By fine-tuning on small amounts of ground truth flow, the network can learn to fill in homogeneous regions and compute full optical flow fields. Using this unsupervised pre-training, our network outperforms similar architectures that were trained supervised using synthetic optical flow.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationUnsupervised Pre-trainingSimilar Papers 제목 키워드 기반
STint: Self-supervised Temporal Interpolation for Geospatial Data
Supervised and unsupervised techniques have demonstrated the potential for temporal interpolation of video data. Nevertheless, most prevailing temporal interpolation techniques hinge on optical flow, which encodes the mo…
Optical Flow EstimationVideo Interpolation and Prediction with Unsupervised Landmarks
Prediction and interpolation for long-range video data involves the complex task of modeling motion trajectories for each visible object, occlusions and dis-occlusions, as well as appearance changes due to viewpoint and …
DecoderMotion InterpolationOptical Flow EstimationPrediction+1FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation
A majority of methods for video frame interpolation compute bidirectional optical flow between adjacent frames of a video, followed by a suitable warping algorithm to generate the output frames. However, approaches relyi…
Action RecognitionMotion MagnificationOptical Flow EstimationVideo Frame InterpolationUnsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling
For visual estimation of optical flow, a crucial function for many vision tasks, unsupervised learning, using the supervision of view synthesis has emerged as a promising alternative to supervised methods, since ground-t…
Optical Flow EstimationUPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learning
We present an unsupervised learning approach for optical flow estimation by improving the upsampling and learning of pyramid network. We design a self-guided upsample module to tackle the interpolation blur problem cause…
Optical Flow Estimation