OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Clouds
In this paper, we study the problem of 3D object segmentation from raw point clouds. Unlike all existing methods which usually require a large amount of human annotations for full supervision, we propose the first unsupervised method, called OGC, to simultaneously identify multiple 3D objects in a single forward pass, without needing any type of human annotations. The key to our approach is to fully leverage the dynamic motion patterns over sequential point clouds as supervision signals to automatically discover rigid objects. Our method consists of three major components, 1) the object segmentation network to directly estimate multi-object masks from a single point cloud frame, 2) the auxiliary self-supervised scene flow estimator, and 3) our core object geometry consistency component. By carefully designing a series of loss functions, we effectively take into account the multi-object rigid consistency and the object shape invariance in both temporal and spatial scales. This allows our method to truly discover the object geometry even in the absence of annotations. We extensively evaluate our method on five datasets, demonstrating the superior performance for object part instance segmentation and general object segmentation in both indoor and the challenging outdoor scenarios.
Code (1)
Tasks
Instance SegmentationObjectSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Unsupervised Volumetric Animation
We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects. Our method learns the 3D structure and dynamics of objects solely from single-view RGB videos, and can decompose them into semant…
3D geometryKeypoint EstimationNovel View SynthesisUnsupervised Monocular Depth Reconstruction of Non-Rigid Scenes
Monocular depth reconstruction of complex and dynamic scenes is a highly challenging problem. While for rigid scenes learning-based methods have been offering promising results even in unsupervised cases, there exists li…
Depth EstimationMotion SegmentationMulti-body SE(3) Equivariance for Unsupervised Rigid Segmentation and Motion Estimation
A truly generalizable approach to rigid segmentation and motion estimation is fundamental to 3D understanding of articulated objects and moving scenes. In view of the closely intertwined relationship between segmentation…
Computational EfficiencyMotion EstimationSegmentationUnsupervised Online 3D Instance Segmentation with Synthetic Sequences and Dynamic Loss
Unsupervised online 3D instance segmentation is a fundamental yet challenging task, as it requires maintaining consistent object identities across LiDAR scans without relying on annotated training data. Existing methods,…
3D Instance SegmentationMoving SLAM: Fully Unsupervised Deep Learning in Non-Rigid Scenes
We propose a method to train deep networks to decompose videos into 3D geometry (camera and depth), moving objects, and their motions, with no supervision. We build on the idea of view synthesis, which uses classical cam…
3D geometryDeep LearningDepth EstimationDepth Prediction+1