Accelerating Video Object Segmentation with Compressed Video
We propose an efficient plug-and-play acceleration framework for semi-supervised video object segmentation by exploiting the temporal redundancies in videos presented by the compressed bitstream. Specifically, we propose a motion vector-based warping method for propagating segmentation masks from keyframes to other frames in a bi-directional and multi-hop manner. Additionally, we introduce a residual-based correction module that can fix wrongly propagated segmentation masks from noisy or erroneous motion vectors. Our approach is flexible and can be added on top of several existing video object segmentation algorithms. We achieved highly competitive results on DAVIS17 and YouTube-VOS on various base models with substantial speed-ups of up to 3.5X with minor drops in accuracy.
Code (2)
Tasks
ObjectSegmentationSemantic SegmentationSemi-Supervised Video Object SegmentationVideo Object SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Multi-Attention Network for Compressed Video Referring Object Segmentation
Referring video object segmentation aims to segment the object referred by a given language expression. Existing works typically require compressed video bitstream to be decoded to RGB frames before being segmented, whic…
ObjectReferring Expression SegmentationReferring Video Object SegmentationSegmentation+3Mimic The Raw Domain: Accelerating Action Recognition in the Compressed Domain
Video understanding usually requires expensive computation that prohibits its deployment, yet videos contain significant spatiotemporal redundancy that can be exploited. In particular, operating directly on the motion ve…
Action RecognitionVideo RecognitionVideo UnderstandingRobust Deep Neural Object Detection and Segmentation for Automotive Driving Scenario with Compressed Image Data
Deep neural object detection or segmentation networks are commonly trained with pristine, uncompressed data. However, in practical applications the input images are usually deteriorated by compression that is applied to …
Autonomous Drivingobject-detectionObject DetectionFast Semantic Segmentation on Video Using Block Motion-Based Feature Interpolation
Convolutional networks optimized for accuracy on challenging, dense prediction tasks are prohibitively slow to run on each frame in a video. The spatial similarity of nearby video frames, however, suggests opportunity to…
Optical Flow EstimationSemantic SegmentationRobust Privacy-Preserving Motion Detection and Object Tracking in Encrypted Streaming Video
Video privacy leakage is becoming an increasingly severe public problem, especially in cloud-based video surveillance systems. It leads to the new need for secure cloud-based video applications, where the video is encryp…
Motion DetectionMoving Object DetectionMultiple Object TrackingObject+5