An object-centric sensitivity analysis of deep learning based instance segmentation
In this study we establish a comprehensive baseline regarding the object-centric robustness of deep learning models for instance segmentation. Our approach is motivated by the work of Geirhos et al. (2019) on texture bias in CNNs. However, we do not compare against human performance but instead incorporate ideas from object-centric representation learning. In addition, we analyze and control the effect of strong stylization that can lead to disappearing objects. The result is a stylized and object-centric version of MS COCO on which we perform an extensive sensitivity analysis regarding visual feature corruptions. We evaluate a broad range of frameworks including Cascade and Mask R-CNN, Swin Transformer, YOLACT(++), DETR, SOTR and SOLOv2. We find that framework choice, data augmentation and dynamic architectures improve robustness whereas supervised and self supervised pre-training does surprisingly not. In summary we evaluate 63 models on 61 versions of COCO for a total of 3843 evaluations.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationInstance SegmentationObjectRepresentation LearningSemantic SegmentationSensitivityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
3D-Aware Instance Segmentation and Tracking in Egocentric Videos
Egocentric videos present unique challenges for 3D scene understanding due to rapid camera motion, frequent object occlusions, and limited object visibility. This paper introduces a novel approach to instance segmentatio…
3D Object ReconstructionInstance SegmentationObjectObject Reconstruction+5HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction
We present HOI4D, a large-scale 4D egocentric dataset with rich annotations, to catalyze the research of category-level human-object interaction. HOI4D consists of 2.4M RGB-D egocentric video frames over 4000 sequences c…
Action SegmentationBenchmarkingHuman-Object Interaction DetectionMotion Segmentation+5Sensitivity of Slot-Based Object-Centric Models to their Number of Slots
Self-supervised methods for learning object-centric representations have recently been applied successfully to various datasets. This progress is largely fueled by slot-based methods, whose ability to cluster visual scen…
SensitivityInstMove: Instance Motion for Object-centric Video Segmentation
Despite significant efforts, cutting-edge video segmentation methods still remain sensitive to occlusion and rapid movement, due to their reliance on the appearance of objects in the form of object embeddings, which are …
ObjectOptical Flow EstimationSegmentationVideo Segmentation+1EgoLifter: Open-world 3D Segmentation for Egocentric Perception
In this paper we present EgoLifter, a novel system that can automatically segment scenes captured from egocentric sensors into a complete decomposition of individual 3D objects. The system is specifically designed for eg…
3D ReconstructionObject