Consistency of augmentation graph and network approximability in contrastive learning
Contrastive learning leverages data augmentation to develop feature representation without relying on large labeled datasets. However, despite its empirical success, the theoretical foundations of contrastive learning remain incomplete, with many essential guarantees left unaddressed, particularly the realizability assumption concerning neural approximability of an optimal spectral contrastive loss solution. In this work, we overcome these limitations by analyzing the pointwise and spectral consistency of the augmentation graph Laplacian. We establish that, under specific conditions for data generation and graph connectivity, as the augmented dataset size increases, the augmentation graph Laplacian converges to a weighted Laplace-Beltrami operator on the natural data manifold. These consistency results ensure that the graph Laplacian spectrum effectively captures the manifold geometry. Consequently, they give way to a robust framework for establishing neural approximability, directly resolving the realizability assumption in a current paradigm.
Code (1)
Tasks
Contrastive LearningData AugmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Generative Contrastive Graph Learning for Recommendation
By treating users' interactions as a user-item graph, graph learning models have been widely deployed in Collaborative Filtering(CF) based recommendation. Recently, researchers have introduced Graph Contrastive Learning(…
Collaborative FilteringContrastive LearningData AugmentationGraph Learning+1Graph-Aware Contrasting for Multivariate Time-Series Classification
Contrastive learning, as a self-supervised learning paradigm, becomes popular for Multivariate Time-Series (MTS) classification. It ensures the consistency across different views of unlabeled samples and then learns effe…
ClassificationContrastive LearningSelf-Supervised LearningTime Series+1LAC: Graph Contrastive Learning with Learnable Augmentation in Continuous Space
Graph Contrastive Learning frameworks have demonstrated success in generating high-quality node representations. The existing research on efficient data augmentation methods and ideal pretext tasks for graph contrastive …
Contrastive LearningData AugmentationDiversityACTIVE:Augmentation-Free Graph Contrastive Learning for Partial Multi-View Clustering
In this paper, we propose an augmentation-free graph contrastive learning framework, namely ACTIVE, to solve the problem of partial multi-view clustering. Notably, we suppose that the representations of similar samples (…
ClusteringContrastive LearningRelationGraph Contrastive Clustering
Recently, some contrastive learning methods have been proposed to simultaneously learn representations and clustering assignments, achieving significant improvements. However, these methods do not take the category infor…
ClusteringContrastive Learning