Human Instance Segmentation and Tracking via Data Association and Single-stage Detector
Human video instance segmentation plays an important role in computer understanding of human activities and is widely used in video processing, video surveillance, and human modeling in virtual reality. Most current VIS methods are based on Mask-RCNN framework, where the target appearance and motion information for data matching will increase computational cost and have an impact on segmentation real-time performance; on the other hand, the existing datasets for VIS focus less on all the people appearing in the video. In this paper, to solve the problems, we develop a new method for human video instance segmentation based on single-stage detector. To tracking the instance across the video, we have adopted data association strategy for matching the same instance in the video sequence, where we jointly learn target instance appearances and their affinities in a pair of video frames in an end-to-end fashion. We have also adopted the centroid sampling strategy for enhancing the embedding extraction ability of instance, which is to bias the instance position to the inside of each instance mask with heavy overlap condition. As a result, even there exists a sudden change in the character activity, the instance position will not move out of the mask, so that the problem that the same instance is represented by two different instances can be alleviated. Finally, we collect PVIS dataset by assembling several video instance segmentation datasets to fill the gap of the current lack of datasets dedicated to human video segmentation. Extensive simulations based on such dataset has been conduct. Simulation results verify the effectiveness and efficiency of the proposed work.
Code (0)
등록된 구현이 없습니다.
Tasks
Human Instance SegmentationInstance SegmentationPositionSegmentationSemantic SegmentationVideo Instance SegmentationVideo SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Multi-Object Tracking and Segmentation with a Space-Time Memory Network
We propose a method for multi-object tracking and segmentation based on a novel memory-based mechanism to associate tracklets. The proposed tracker, MeNToS, addresses particularly the long-term data association problem, …
Instance SegmentationMulti-Object TrackingMulti-Object Tracking and SegmentationMultiple Object Tracking+9STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation
Video Instance Segmentation (VIS) is a task that simultaneously requires classification, segmentation, and instance association in a video. Recent VIS approaches rely on sophisticated pipelines to achieve this goal, incl…
Contrastive LearningInstance SegmentationSegmentationSemantic Segmentation+1Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Clouds
Robots and autonomous vehicles should be aware of what happens in their surroundings. The segmentation and tracking of moving objects are essential for reliable path planning, including collision avoidance. We investigat…
Collision AvoidanceAutonomous VehiclesPoint CloudsMeNToS: Tracklets Association with a Space-Time Memory Network
We propose a method for multi-object tracking and segmentation (MOTS) that does not require fine-tuning or per benchmark hyperparameter selection. The proposed method addresses particularly the data association problem. …
Instance SegmentationMulti-Object TrackingMulti-Object Tracking and SegmentationObject+6PointTrack++ for Effective Online Multi-Object Tracking and Segmentation
Multiple-object tracking and segmentation (MOTS) is a novel computer vision task that aims to jointly perform multiple object tracking (MOT) and instance segmentation. In this work, we present PointTrack++, an effective …
Data AugmentationDecoderInstance SegmentationMulti-Object Tracking+7