Negative Sampling for Contrastive Representation Learning: A Review
The learn-to-compare paradigm of contrastive representation learning (CRL), which compares positive samples with negative ones for representation learning, has achieved great success in a wide range of domains, including natural language processing, computer vision, information retrieval and graph learning. While many research works focus on data augmentations, nonlinear transformations or other certain parts of CRL, the importance of negative sample selection is usually overlooked in literature. In this paper, we provide a systematic review of negative sampling (NS) techniques and discuss how they contribute to the success of CRL. As the core part of this paper, we summarize the existing NS methods into four categories with pros and cons in each genre, and further conclude with several open research questions as future directions. By generalizing and aligning the fundamental NS ideas across multiple domains, we hope this survey can accelerate cross-domain knowledge sharing and motivate future researches for better CRL.
Code (0)
등록된 구현이 없습니다.
Tasks
Graph LearningInformation RetrievalRepresentation LearningRetrievalSimilar Papers 제목 키워드 기반
ConCur: Self-supervised graph representation based on contrastive learning with curriculum negative sampling
Contrastive learning has made breakthrough advancements in graph representation learning, which encourages the representation of positive samples to be close and those of negative samples to be far away. However, existin…
Contrastive LearningGraph Representation LearningNode ClassificationRepresentation Learning+1Contrastive Learning with Hard Negative Samples
How can you sample good negative examples for contrastive learning? We argue that, as with metric learning, contrastive learning of representations benefits from hard negative samples (i.e., points that are difficult to …
Contrastive LearningMetric LearningContrastive Learning with Negative Sampling Correction
As one of the most effective self-supervised representation learning methods, contrastive learning (CL) relies on multiple negative pairs to contrast against each positive pair. In the standard practice of contrastive le…
Contrastive LearningData AugmentationGraph ClassificationRepresentation LearningHard Negative Sampling Strategies for Contrastive Representation Learning
One of the challenges in contrastive learning is the selection of appropriate \textit{hard negative} examples, in the absence of label information. Random sampling or importance sampling methods based on feature similari…
Contrastive LearningRepresentation LearningHard Negative Sampling via Regularized Optimal Transport for Contrastive Representation Learning
We study the problem of designing hard negative sampling distributions for unsupervised contrastive representation learning. We propose and analyze a novel min-max framework that seeks a representation which minimizes th…
Contrastive LearningRepresentation Learning