Temporally Robust Global Motion Compensation by Keypoint-based Congealing
Global motion compensation (GMC) removes the impact of camera motion and creates a video in which the background appears static over the progression of time. Various vision problems, such as human activity recognition, background reconstruction, and multi-object tracking can benefit from GMC. Existing GMC algorithms rely on sequentially processing consecutive frames, by estimating the transformation mapping the two frames, and obtaining a composite transformation to a global motion compensated coordinate. Sequential GMC suffers from temporal drift of frames from the accurate global coordinate, due to either error accumulation or sporadic failures of motion estimation at a few frames. We propose a temporally robust global motion compensation (TRGMC) algorithm which performs accurate and stable GMC, despite complicated and long-term camera motion. TRGMC densely connects pairs of frames, by matching local keypoints of each frame. A joint alignment of these frames is formulated as a novel keypoint-based congealing problem, where the transformation of each frame is updated iteratively, such that the spatial coordinates for the start and end points of matched keypoints are identical. Experimental results demonstrate that TRGMC has superior performance in a wide range of scenarios.
Code (0)
등록된 구현이 없습니다.
Tasks
Activity RecognitionHuman Activity RecognitionMotion CompensationMotion EstimationMulti-Object TrackingObject TrackingSimilar Papers 제목 키워드 기반
LVC-LGMC: Joint Local and Global Motion Compensation for Learned Video Compression
Existing learned video compression models employ flow net or deformable convolutional networks (DCN) to estimate motion information. However, the limited receptive fields of flow net and DCN inherently direct their atten…
Motion CompensationVideo CompressionViewpoint distortion compensation in practical surveillance systems
Our aim is to estimate the perspective-effected geometric distortion of a scene from a video feed. In contrast to all previous work we wish to achieve this using from low-level, spatio-temporally local motion features us…
global-optimizationImplicit Motion-Compensated Network for Unsupervised Video Object Segmentation
Unsupervised video object segmentation (UVOS) aims at automatically separating the primary foreground object(s) from the background in a video sequence. Existing UVOS methods either lack robustness when there are visuall…
Motion CompensationSemantic SegmentationUnsupervised Video Object SegmentationVideo Object Segmentation+1Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation
Talking head video generation aims to animate a human face in a still image with dynamic poses and expressions using motion information derived from a target-driving video, while maintaining the person's identity in the …
Talking Head GenerationVideo GenerationSuperEvent: Cross-Modal Learning of Event-based Keypoint Detection
Event-based keypoint detection and matching holds significant potential, enabling the integration of event sensors into highly optimized Visual SLAM systems developed for frame cameras over decades of research. Unfortuna…
Keypoint Detection