Contrastive Learning Is Spectral Clustering On Similarity Graph

2023-03-27 · Zhiquan Tan, Yifan Zhang, Jingqin Yang, Yang Yuan

Contrastive learning is a powerful self-supervised learning method, but we have a limited theoretical understanding of how it works and why it works. In this paper, we prove that contrastive learning with the standard InfoNCE loss is equivalent to spectral clustering on the similarity graph. Using this equivalence as the building block, we extend our analysis to the CLIP model and rigorously characterize how similar multi-modal objects are embedded together. Motivated by our theoretical insights, we introduce the Kernel-InfoNCE loss, incorporating mixtures of kernel functions that outperform the standard Gaussian kernel on several vision datasets. The code is available at https://github.com/yifanzhang-pro/Kernel-InfoNCE.

📄 PDF Abstract BibTeX arXiv:2303.15103

Code (1)

yifanzhang-pro/kernel-infonce 공식 구현 pytorch

Tasks

ClusteringContrastive Learning Self-Supervised Learning

Methods 이 논문이 사용한 방법론

Spectral Clustering Spectral clustering has attracted increasing attention due to the promising ability in dealing with nonlinearly separable datasets [15], [16]. In spectral clustering, the…

CLIP Contrastive Language-Image Pre-training (CLIP), consisting of a simplified version of ConVIRT trained from scratch, is an efficient method of image representation learning…

Contrastive Learning 설명 없음

InfoNCE 설명 없음

UMAP Is Spectral Clustering on the Fuzzy Nearest-Neighbor Graph

2026-02-12 · Yang Yang arxiv

UMAP (Uniform Manifold Approximation and Projection) is among the most widely used algorithms for non linear dimensionality reduction and data visualisation. Despite its popularity, and despite being presented through th…

Dimensionality Reduction Contrastive Learning

Superpixel Graph Contrastive Clustering with Semantic-Invariant Augmentations for Hyperspectral Images

2024-03-04 · Jianhan Qi, Yuheng Jia, Hui Liu, Junhui Hou

Hyperspectral images (HSI) clustering is an important but challenging task. The state-of-the-art (SOTA) methods usually rely on superpixels, however, they do not fully utilize the spatial and spectral information in HSI …

Clustering Contrastive Learning Superpixels

Towards Multi-view Graph Anomaly Detection with Similarity-Guided Contrastive Clustering

2024-09-15 · Lecheng Zheng, John R. Birge, Yifang Zhang, Jingrui He

Anomaly detection on graphs plays an important role in many real-world applications. Usually, these data are composed of multiple types (e.g., user information and transaction records for financial data), thus exhibiting…

Anomaly Detection Clustering Contrastive Learning Graph Anomaly Detection

FedSpectral+: Spectral Clustering using Federated Learning

2023-02-04 · Janvi Thakkar, Devvrat Joshi

Clustering in graphs has been a well-known research problem, particularly because most Internet and social network data is in the form of graphs. Organizations widely use spectral clustering algorithms to find clustering…

Clustering Federated Learning

Large-Scale Hyperspectral Image Clustering Using Contrastive Learning

2021-11-15 · Yaoming Cai, Zijia Zhang, Yan Liu, Pedram Ghamisi 외

Clustering of hyperspectral images is a fundamental but challenging task. The recent development of hyperspectral image clustering has evolved from shallow models to deep and achieved promising results in many benchmark …

Clustering Contrastive Learning hyperspectral image clustering Image Clustering+2