Temporally Distributed Networks for Fast Video Semantic Segmentation
We present TDNet, a temporally distributed network designed for fast and accurate video semantic segmentation. We observe that features extracted from a certain high-level layer of a deep CNN can be approximated by composing features extracted from several shallower sub-networks. Leveraging the inherent temporal continuity in videos, we distribute these sub-networks over sequential frames. Therefore, at each time step, we only need to perform a lightweight computation to extract a sub-features group from a single sub-network. The full features used for segmentation are then recomposed by application of a novel attention propagation module that compensates for geometry deformation between frames. A grouped knowledge distillation loss is also introduced to further improve the representation power at both full and sub-feature levels. Experiments on Cityscapes, CamVid, and NYUD-v2 demonstrate that our method achieves state-of-the-art accuracy with significantly faster speed and lower latency.
Code (1)
Tasks
Knowledge DistillationReal-Time Semantic SegmentationSegmentationSemantic SegmentationVideo Semantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
One-Shot Video Object Segmentation
This paper tackles the task of semi-supervised video object segmentation, i.e., the separation of an object from the background in a video, given the mask of the first frame. We present One-Shot Video Object Segmentation…
Foreground SegmentationObjectSegmentationSemi-Supervised Video Object Segmentation+3Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation
Action segmentation refers to inferring boundaries of semantically consistent visual concepts in videos and is an important requirement for many video understanding tasks. For this and other video understanding tasks, su…
Action SegmentationClusteringSegmentationUnsupervised Action Segmentation+1MonoDVPS: A Self-Supervised Monocular Depth Estimation Approach to Depth-aware Video Panoptic Segmentation
Depth-aware video panoptic segmentation tackles the inverse projection problem of restoring panoptic 3D point clouds from video sequences, where the 3D points are augmented with semantic classes and temporally consistent…
Depth-aware Video Panoptic SegmentationDepth EstimationDepth PredictionImage Segmentation+5Temporally Constrained Neural Networks (TCNN): A framework for semi-supervised video semantic segmentation
A major obstacle to building models for effective semantic segmentation, and particularly video semantic segmentation, is a lack of large and well annotated datasets. This bottleneck is particularly prohibitive in highly…
SegmentationSemantic SegmentationVideo Semantic SegmentationVidSeg: Training-free Video Semantic Segmentation based on Diffusion Models
We introduce the first training-free approach for Video Semantic Segmentation (VSS) based on pre-trained diffusion models. A growing research direction attempts to employ diffusion models to perform downstream vision…
SegmentationSemantic SegmentationVideo Semantic Segmentation