A Closer Look at Invariances in Self-supervised Pre-training for 3D Vision
Self-supervised pre-training for 3D vision has drawn increasing research interest in recent years. In order to learn informative representations, a lot of previous works exploit invariances of 3D features, e.g., perspective-invariance between views of the same scene, modality-invariance between depth and RGB images, format-invariance between point clouds and voxels. Although they have achieved promising results, previous researches lack a systematic and fair comparison of these invariances. To address this issue, our work, for the first time, introduces a unified framework, under which various pre-training methods can be investigated. We conduct extensive experiments and provide a closer look at the contributions of different invariances in 3D pre-training. Also, we propose a simple but effective method that jointly pre-trains a 3D encoder and a depth map encoder using contrastive learning. Models pre-trained with our method gain significant performance boost in downstream tasks. For instance, a pre-trained VoteNet outperforms previous methods on SUN RGB-D and ScanNet object detection benchmarks with a clear margin.
Code (1)
Tasks
Contrastive Learningobject-detectionObject DetectionSimilar Papers 제목 키워드 기반
Self-Supervised Learning of Video-Induced Visual Invariances
We propose a general framework for self-supervised learning of transferable visual representations based on Video-Induced Visual Invariances (VIVI). We consider the implicit hierarchy present in the videos and make use o…
Image ClassificationSelf-Supervised LearningTransfer LearningDo Invariances in Deep Neural Networks Align with Human Perception?
An evaluation criterion for safe and trustworthy deep learning is how well the invariances captured by representations of deep neural networks (DNNs) are shared with humans. We identify challenges in measuring these inva…
Data AugmentationSelf-Supervised LearningLearning Online Visual Invariances for Novel Objects via Supervised and Self-Supervised Training
Humans can identify objects following various spatial transformations such as scale and viewpoint. This extends to novel objects, after a single presentation at a single pose, sometimes referred to as online invariance. …
Data AugmentationTranslationAmortised Invariance Learning for Contrastive Self-Supervision
Contrastive self-supervised learning methods famously produce high quality transferable representations by learning invariances to different data augmentations. Invariances established during pre-training can be interpre…
Contrastive LearningRepresentation LearningSelf-Supervised LearningA Theoretical Characterization of Optimal Data Augmentations in Self-Supervised Learning
Data augmentations play an important role in the recent success of Self-Supervised Learning (SSL). While commonly viewed as encoding invariances into the learned representations, this interpretation overlooks the impact …
Self-Supervised Learning