Joint Unsupervised Deformable Spatio-Temporal Alignment of Sequences
Typically, the problems of spatial and temporal alignment of sequences are considered disjoint. That is, in order to align two sequences, a methodology that (non)-rigidly aligns the images is first applied, followed by temporal alignment of the obtained aligned images. In this paper, we propose the first, to the best of our knowledge, methodology that can jointly spatio-temporally align two sequences, which display highly deformable texture-varying objects. We show that by treating the problems of deformable spatial and temporal alignment jointly, we achieve better results than considering the problems independent. Furthermore, we show that deformable spatio-temporal alignment of faces can be performed in an unsupervised manner (i.e., without employing face trackers or building person-specific deformable models).
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Cross-Modal Learning with 3D Deformable Attention for Action Recognition
An important challenge in vision-based action recognition is the embedding of spatiotemporal features with two or more heterogeneous modalities into a single feature. In this study, we propose a new 3D deformable transfo…
Action RecognitionSpatio-temporal deformable convolution for compressed video quality enhancement
Recent years have witnessed remarkable success of deep learning methods in quality enhancement for compressed video. To better explore temporal information, existing methods usually estimate optical flow for temporal mot…
Video EnhancementVideo RestorationDeformable 3D Convolution for Video Super-Resolution
The spatio-temporal information among video sequences is significant for video super-resolution (SR). However, the spatio-temporal information cannot be fully used by existing video SR methods since spatial feature extra…
Motion CompensationSuper-ResolutionVideo Super-ResolutionSpatiotemporal Deformable Part Models for Action Detection
Deformable part models have achieved impressive performance for object detection, even on difficult image datasets. This paper explores the generalization of deformable part models from 2D images to 3D spatiotemporal vol…
Action Detectionobject-detectionObject DetectionSpatio-Temporal Deformable Attention Network for Video Deblurring
The key success factor of the video deblurring methods is to compensate for the blurry pixels of the mid-frame with the sharp pixels of the adjacent video frames. Therefore, mainstream methods align the adjacent frames b…
DeblurringDecoderVideo Deblurring