Learning Video Stabilization Using Optical Flow
We propose a novel neural network that infers the per-pixel warp fields for video stabilization from the optical flow fields of the input video. While previous learning based video stabilization methods attempt to implicitly learn frame motions from color videos, our method resorts to optical flow for motion analysis and directly learns the stabilization using the optical flow. We also propose a pipeline that uses optical flow principal components for motion inpainting and warp field smoothing, making our method robust to moving objects, occlusion and optical flow inaccuracy, which is challenging for other video stabilization methods. Our method achieves quantitatively and visually better results than the state-of-the-art optimization based and deep learning based video stabilization methods. Our method also gives a 3x speed improvement compared to the optimization based methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Optical Flow EstimationVideo StabilizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SteadyFlow: Spatially Smooth Optical Flow for Video Stabilization
We propose a novel motion model, SteadyFlow, to represent the motion between neighboring video frames for stabilization. A SteadyFlow is a specific optical flow by enforcing strong spatial coherence, such that smoothing …
Optical Flow EstimationVideo StabilizationReal-Time Optical flow-based Video Stabilization for Unmanned Aerial Vehicles
This paper describes the development of a novel algorithm to tackle the problem of real-time video stabilization for unmanned aerial vehicles (UAVs). There are two main components in the algorithm: (1) By designing a sui…
Optical Flow EstimationVideo StabilizationRobust Video Stabilization by Optimization in CNN Weight Space
We propose a novel robust video stabilization method. Unlike traditional video stabilization techniques that involve complex motion models, we directly model the appearance change of the frames as the dense optical flow …
Optical Flow EstimationVideo StabilizationGlobalFlowNet: Video Stabilization using Deep Distilled Global Motion Estimates
Videos shot by laymen using hand-held cameras contain undesirable shaky motion. Estimating the global motion between successive frames, in a manner not influenced by moving objects, is central to many video stabilization…
Knowledge DistillationOptical Flow EstimationVideo StabilizationHybrid Neural Fusion for Full-frame Video Stabilization
Existing video stabilization methods often generate visible distortion or require aggressive cropping of frame boundaries, resulting in smaller field of views. In this work, we present a frame synthesis algorithm to achi…
Optical Flow EstimationVideo Stabilization