Self-Supervised Multi-View Learning via Auto-Encoding 3D Transformations
3D object representation learning is a fundamental challenge in computer vision to infer about the 3D world. Recent advances in deep learning have shown their efficiency in 3D object recognition, among which view-based methods have performed best so far. However, feature learning of multiple views in existing methods is mostly performed in a supervised fashion, which often requires a large amount of data labels with high costs. In contrast, self-supervised learning aims to learn multi-view feature representations without involving labeled data. To this end, we propose a novel self-supervised paradigm to learn Multi-View Transformation Equivariant Representations (MV-TER), exploring the equivariant transformations of a 3D object and its projected multiple views. Specifically, we perform a 3D transformation on a 3D object, and obtain multiple views before and after the transformation via projection. Then, we self-train a representation to capture the intrinsic 3D object representation by decoding 3D transformation parameters from the fused feature representations of multiple views before and after the transformation. Experimental results demonstrate that the proposed MV-TER significantly outperforms the state-of-the-art view-based approaches in 3D object classification and retrieval tasks, and show the generalization to real-world datasets.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Object Classification3D Object RecognitionMULTI-VIEW LEARNINGObjectObject RecognitionRepresentation LearningRetrievalSelf-Supervised LearningSimilar Papers 제목 키워드 기반
RayZer: A Self-supervised Large View Synthesis Model
We present RayZer, a self-supervised multi-view 3D Vision model trained without any 3D supervision, i.e., camera poses and scene geometry, while exhibiting emerging 3D awareness. Concretely, RayZer takes unposed and unca…
modelNovel View SynthesisSocially-Aware Self-Supervised Tri-Training for Recommendation
Self-supervised learning (SSL), which can automatically generate ground-truth samples from raw data, holds vast potential to improve recommender systems. Most existing SSL-based methods perturb the raw data graph with un…
Contrastive LearningRecommendation SystemsRepresentation LearningSelf-Supervised LearningContrastive Multi-View Representation Learning on Graphs
We introduce a self-supervised approach for learning node and graph level representations by contrasting structural views of graphs. We show that unlike visual representation learning, increasing the number of views to m…
General ClassificationGraph ClassificationLinear evaluationRepresentation Learning+1SatDINO: A Deep Dive into Self-Supervised Pretraining for Remote Sensing
Self-supervised learning has emerged as a powerful tool for remote sensing, where large amounts of unlabeled data are available. In this work, we investigate the use of DINO, a contrastive self-supervised method, for pre…
Self-Supervised LearningRepresentation LearningAASAE: Augmentation-Augmented Stochastic Autoencoders
Recent methods for self-supervised learning can be grouped into two paradigms: contrastive and non-contrastive approaches. Their success can largely be attributed to data augmentation pipelines which generate multiple vi…
Contrastive LearningData Augmentationimage-classificationImage Classification+1