Efficient Video Semantic Segmentation with Labels Propagation and Refinement
This paper tackles the problem of real-time semantic segmentation of high definition videos using a hybrid GPU / CPU approach. We propose an Efficient Video Segmentation(EVS) pipeline that combines: (i) On the CPU, a very fast optical flow method, that is used to exploit the temporal aspect of the video and propagate semantic information from one frame to the next. It runs in parallel with the GPU. (ii) On the GPU, two Convolutional Neural Networks: A main segmentation network that is used to predict dense semantic labels from scratch, and a Refiner that is designed to improve predictions from previous frames with the help of a fast Inconsistencies Attention Module (IAM). The latter can identify regions that cannot be propagated accurately. We suggest several operating points depending on the desired frame rate and accuracy. Our pipeline achieves accuracy levels competitive to the existing real-time methods for semantic image segmentation(mIoU above 60%), while achieving much higher frame rates. On the popular Cityscapes dataset with high resolution frames (2048 x 1024), the proposed operating points range from 80 to 1000 Hz on a single GPU and CPU.
Code (0)
등록된 구현이 없습니다.
Tasks
CPUGPUImage SegmentationOptical Flow EstimationPlaying the Game of 2048Real-Time Semantic SegmentationSegmentationSemantic SegmentationVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
RePL: Pseudo-label Refinement for Semi-supervised LiDAR Semantic Segmentation
Semi-supervised learning for LiDAR semantic segmentation often suffers from error propagation and confirmation bias caused by noisy pseudo-labels. To tackle this chronic issue, we introduce RePL, a novel framework that e…
LIDAR Semantic SegmentationContext Propagation from Proposals for Semantic Video Object Segmentation
In this paper, we propose a novel approach to learning semantic contextual relationships in videos for semantic object segmentation. Our algorithm derives the semantic contexts from video object proposals which encode th…
ObjectSegmentationSemantic SegmentationSuperpixels+2Video Propagation Networks
We propose a technique that propagates information forward through video data. The method is conceptually simple and can be applied to tasks that require the propagation of structured information, such as semantic labels…
SegmentationSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object Segmentation+4Image Diffusion Models Exhibit Emergent Temporal Propagation in Videos
Image diffusion models, though originally developed for image generation, implicitly capture rich semantic structures that enable various recognition and localization tasks beyond synthesis. In this work, we investigate …
Video Object SegmentationImage GenerationObject TrackingImproving Semantic Segmentation via Video Propagation and Label Relaxation
Semantic segmentation requires large amounts of pixel-wise annotations to learn accurate models. In this paper, we present a video prediction-based methodology to scale up training sets by synthesizing new training sampl…
SegmentationSemantic SegmentationVideo Propagation