Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrast
Geometry and color information provided by the point clouds are both crucial for 3D scene understanding. Two pieces of information characterize the different aspects of point clouds, but existing methods lack an elaborate design for the discrimination and relevance. Hence we explore a 3D self-supervised paradigm that can better utilize the relations of point cloud information. Specifically, we propose a universal 3D scene pre-training framework via Geometry-Color Contrast (Point-GCC), which aligns geometry and color information using a Siamese network. To take care of actual application tasks, we design (i) hierarchical supervision with point-level contrast and reconstruct and object-level contrast based on the novel deep clustering module to close the gap between pre-training and downstream tasks; (ii) architecture-agnostic backbone to adapt for various downstream models. Benefiting from the object-level representation associated with downstream tasks, Point-GCC can directly evaluate model performance and the result demonstrates the effectiveness of our methods. Transfer learning results on a wide range of tasks also show consistent improvements across all datasets. e.g., new state-of-the-art object detection results on SUN RGB-D and S3DIS datasets. Codes will be released at https://github.com/Asterisci/Point-GCC.
Code (1)
Tasks
3D Instance Segmentation3D Object Detection3D Semantic SegmentationDeep ClusteringObjectobject-detectionObject DetectionScene UnderstandingTransfer LearningUnsupervised 3D Semantic SegmentationSimilar Papers 제목 키워드 기반
RigidFlow: Self-Supervised Scene Flow Learning on Point Clouds by Local Rigidity Prior
In this work, we focus on scene flow learning on point clouds in a self-supervised manner. A real-world scene can be well modeled as a collection of rigidly moving parts, therefore its scene flow can be represented a…
Motion EstimationSelf-Supervised LearningAdversarial Self-Supervised Scene Flow Estimation
This work proposes a metric learning approach for self-supervised scene flow estimation. Scene flow estimation is the task of estimating 3D flow vectors for consecutive 3D point clouds. Such flow vectors are fruitful, \e…
Metric LearningScene Flow EstimationSelf-supervised Scene Flow EstimationTripletUniVIP: A Unified Framework for Self-Supervised Visual Pre-training
Self-supervised learning (SSL) holds promise in leveraging large amounts of unlabeled data. However, the success of popular SSL methods has limited on single-centric-object images like those in ImageNet and ignores the c…
image-classificationImage ClassificationObjectobject-detection+3Block-to-Scene Pre-training for Point Cloud Hybrid-Domain Masked Autoencoders
Point clouds, as a primary representation of 3D data, can be categorized into scene domain point clouds and object domain point clouds based on the modeled content. Masked autoencoders (MAE) have become the mainstream pa…
ObjectPosition regressionSelf-Supervised LearningSelf-Supervised Learning of Non-Rigid Residual Flow and Ego-Motion
Most of the current scene flow methods choose to model scene flow as a per point translation vector without differentiating between static and dynamic components of 3D motion. In this work we present an alternative metho…
Self-Supervised LearningTranslation