SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption
Self-supervised contrastive representation learning has proved incredibly successful in the vision and natural language domains, enabling state-of-the-art performance with orders of magnitude less labeled data. However, such methods are domain-specific and little has been done to leverage this technique on real-world tabular datasets. We propose SCARF, a simple, widely-applicable technique for contrastive learning, where views are formed by corrupting a random subset of features. When applied to pre-train deep neural networks on the 69 real-world, tabular classification datasets from the OpenML-CC18 benchmark, SCARF not only improves classification accuracy in the fully-supervised setting but does so also in the presence of label noise and in the semi-supervised setting where only a fraction of the available training data is labeled. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders. We conduct comprehensive ablations, detailing the importance of a range of factors.
Code (1)
Tasks
Contrastive LearningRepresentation Learningtabular-classificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Data
Medical datasets and especially biobanks, often contain extensive tabular data with rich clinical information in addition to images. In practice, clinicians typically have less data, both in terms of diversity and scale,…
Contrastive LearningISD: Self-Supervised Learning by Iterative Similarity Distillation
Recently, contrastive learning has achieved great results in self-supervised learning, where the main idea is to push two augmentations of an image (positive pairs) closer compared to other random images (negative pairs)…
Contrastive LearningSelf-Supervised LearningTransfer LearningGraphCL: Contrastive Self-Supervised Learning of Graph Representations
We propose Graph Contrastive Learning (GraphCL), a general framework for learning node representations in a self supervised manner. GraphCL learns node embeddings by maximizing the similarity between the representations …
Contrastive LearningInductive LearningNode ClassificationSelf-Supervised LearningUnsupervised Feature Learning for Manipulation with Contrastive Domain Randomization
Robotic tasks such as manipulation with visual inputs require image features that capture the physical properties of the scene, e.g., the position and configuration of objects. Recently, it has been suggested to learn su…
Random Walks in Self-supervised Learning for Triangular Meshes
This study addresses the challenge of self-supervised learning for 3D mesh analysis. It presents an new approach that uses random walks as a form of data augmentation to generate diverse representations of mesh surfaces.…
ClusteringContrastive LearningData AugmentationRetrieval+1