Contrastive learning, multi-view redundancy, and linear models
Self-supervised learning is an empirically successful approach to unsupervised learning based on creating artificial supervised learning problems. A popular self-supervised approach to representation learning is contrastive learning, which leverages naturally occurring pairs of similar and dissimilar data points, or multiple views of the same data. This work provides a theoretical analysis of contrastive learning in the multi-view setting, where two views of each datum are available. The main result is that linear functions of the learned representations are nearly optimal on downstream prediction tasks whenever the two views provide redundant information about the label.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningRepresentation LearningSelf-Supervised LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy
In a wide range of multimodal tasks, contrastive learning has become a particularly appealing approach since it can successfully learn representations from abundant unlabeled data with only pairing information (e.g., ima…
Contrastive LearningRepresentation LearningDual Adversarial Perturbators Generate rich Views for Recommendation
Graph contrastive learning (GCL) has been extensively studied and leveraged as a potent tool in recommender systems. Most existing GCL-based recommenders generate contrastive views by altering the graph structure or intr…
Contrastive LearningGraph LearningRecommendation SystemsMetaMask: Revisiting Dimensional Confounder for Self-Supervised Learning
As a successful approach to self-supervised learning, contrastive learning aims to learn invariant information shared among distortions of the input sample. While contrastive learning has yielded continuous advancements …
Contrastive LearningMeta-LearningSelf-Supervised LearningKernel Alignment-based Multi-view Unsupervised Feature Selection with Sample-level Adaptive Graph Learning
Although multi-view unsupervised feature selection (MUFS) has demonstrated success in dimensionality reduction for unlabeled multi-view data, most existing methods reduce feature redundancy by focusing on linear correlat…
Dimensionality ReductionGraph LearningGPS: Graph Contrastive Learning via Multi-scale Augmented Views from Adversarial Pooling
Self-supervised graph representation learning has recently shown considerable promise in a range of fields, including bioinformatics and social networks. A large number of graph contrastive learning approaches have shown…
Adversarial RobustnessContrastive LearningGraph ClassificationGraph Representation Learning+2