pix2gestalt: Amodal Segmentation by Synthesizing Wholes
We introduce pix2gestalt, a framework for zero-shot amodal segmentation, which learns to estimate the shape and appearance of whole objects that are only partially visible behind occlusions. By capitalizing on large-scale diffusion models and transferring their representations to this task, we learn a conditional diffusion model for reconstructing whole objects in challenging zero-shot cases, including examples that break natural and physical priors, such as art. As training data, we use a synthetically curated dataset containing occluded objects paired with their whole counterparts. Experiments show that our approach outperforms supervised baselines on established benchmarks. Our model can furthermore be used to significantly improve the performance of existing object recognition and 3D reconstruction methods in the presence of occlusions.
Code (1)
Tasks
3D ReconstructionObject RecognitionSegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Finding Closure: A Closer Look at the Gestalt Law of Closure in Convolutional Neural Networks
The human brain has an inherent ability to fill in gaps to perceive figures as complete wholes, even when parts are missing or fragmented. This phenomenon is known as Closure in psychology, one of the Gestalt laws of per…
Object RecognitionLearning to See the Invisible: End-to-End Trainable Amodal Instance Segmentation
Semantic amodal segmentation is a recently proposed extension to instance-aware segmentation that includes the prediction of the invisible region of each object instance. We present the first all-in-one end-to-end traina…
Amodal Instance SegmentationData AugmentationInstance SegmentationSegmentation+1Foundation Models for Amodal Video Instance Segmentation in Automated Driving
In this work, we study amodal video instance segmentation for automated driving. Previous works perform amodal video instance segmentation relying on methods trained on entirely labeled video data with techniques borrowe…
Amodal Instance SegmentationInstance SegmentationPoint TrackingSegmentation+2Amodal Ground Truth and Completion in the Wild
This paper studies amodal image segmentation: predicting entire object segmentation masks including both visible and invisible (occluded) parts. In previous work, the amodal segmentation ground truth on real images is us…
Image SegmentationSegmentationSemantic SegmentationCoarse-to-Fine Amodal Segmentation with Shape Prior
Amodal object segmentation is a challenging task that involves segmenting both visible and occluded parts of an object. In this paper, we propose a novel approach, called Coarse-to-Fine Segmentation (C2F-Seg), that addre…
ObjectSegmentationSemantic Segmentation