Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework
As a seminal tool in self-supervised representation learning, contrastive learning has gained unprecedented attention in recent years. In essence, contrastive learning aims to leverage pairs of positive and negative samples for representation learning, which relates to exploiting neighborhood information in a feature space. By investigating the connection between contrastive learning and neighborhood component analysis (NCA), we provide a novel stochastic nearest neighbor viewpoint of contrastive learning and subsequently propose a series of contrastive losses that outperform the existing ones. Under our proposed framework, we show a new methodology to design integrated contrastive losses that could simultaneously achieve good accuracy and robustness on downstream tasks. With the integrated framework, we achieve up to 6\% improvement on the standard accuracy and 17\% improvement on the robust accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Revisiting local branching with a machine learning lens
Finding high-quality solutions to mixed-integer linear programming problems (MILPs) is of great importance for many practical applications. In this respect, the refinement heuristic local branching (LB) has been proposed…
BIG-bench Machine LearningThrough the LENS: Local Geometric Decomposition of Vision-Language Model Representations
Vision-language models (VLMs) process image patches and text tokens in a shared residual stream, but the local geometry through which the two modalities interact remains poorly understood. Most interpretability methods i…
Text RetrievalRevisiting Neighborhood Aggregation in Graph Neural Networks for Node Classification using Statistical Signal Processing
We delve into the issue of node classification within graphs, specifically reevaluating the concept of neighborhood aggregation, which is a fundamental component in graph neural networks (GNNs). Our analysis reveals conc…
ClassificationNode ClassificationNeighborhood-Enhanced Supervised Contrastive Learning for Collaborative Filtering
While effective in recommendation tasks, collaborative filtering (CF) techniques face the challenge of data sparsity. Researchers have begun leveraging contrastive learning to introduce additional self-supervised signals…
Collaborative FilteringContrastive LearningData AugmentationRecommendation Systems