Robust High-Resolution Video Matting with Temporal Guidance
We introduce a robust, real-time, high-resolution human video matting method that achieves new state-of-the-art performance. Our method is much lighter than previous approaches and can process 4K at 76 FPS and HD at 104 FPS on an Nvidia GTX 1080Ti GPU. Unlike most existing methods that perform video matting frame-by-frame as independent images, our method uses a recurrent architecture to exploit temporal information in videos and achieves significant improvements in temporal coherence and matting quality. Furthermore, we propose a novel training strategy that enforces our network on both matting and segmentation objectives. This significantly improves our model's robustness. Our method does not require any auxiliary inputs such as a trimap or a pre-captured background image, so it can be widely applied to existing human matting applications.
Code (1)
Tasks
4kGPUImage MattingVideo MattingVocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Ultrahigh Resolution Image/Video Matting With Spatio-Temporal Sparsity
Commodity ultra-high definition (UHD) displays are becoming more affordable which demand imaging in ultra high resolution (UHR). This paper proposes SparseMat, a computationally efficient approach for UHR image/video…
Image MattingVideo MattingDeep Video Matting via Spatio-Temporal Alignment and Aggregation
Despite the significant progress made by deep learning in natural image matting, there has been so far no representative work on deep learning for video matting due to the inherent technical challenges in reasoning tempo…
DecoderDeep LearningImage MattingOptical Flow Estimation+1Video Instance Matting
Conventional video matting outputs one alpha matte for all instances appearing in a video frame so that individual instances are not distinguished. While video instance segmentation provides time-consistent instance mask…
BinarizationImage MattingInstance SegmentationSemantic Segmentation+2Attention-guided Temporally Coherent Video Object Matting
This paper proposes a novel deep learning-based video object matting method that can achieve temporally coherent matting results. Its key component is an attention-based temporal aggregation module that maximizes image m…
Image MattingObjectSemantic SegmentationVideo Matting+2Real-Time High-Resolution Background Matting
We introduce a real-time, high-resolution background replacement technique which operates at 30fps in 4K resolution, and 60fps for HD on a modern GPU. Our technique is based on background matting, where an additional fra…
4kGPUImage MattingVocal Bursts Intensity Prediction