Multi-level Supervised Contrastive Learning
Contrastive learning is a well-established paradigm in representation learning. The standard framework of contrastive learning minimizes the distance between "similar" instances and maximizes the distance between dissimilar ones in the projection space, disregarding the various aspects of similarity that can exist between two samples. Current methods rely on a single projection head, which fails to capture the full complexity of different aspects of a sample, leading to suboptimal performance, especially in scenarios with limited training data. In this paper, we present a novel supervised contrastive learning method in a unified framework called multilevel contrastive learning (MLCL), that can be applied to both multi-label and hierarchical classification tasks. The key strength of the proposed method is the ability to capture similarities between samples across different labels and/or hierarchies using multiple projection heads. Extensive experiments on text and image datasets demonstrate that the proposed approach outperforms state-of-the-art contrastive learning methods
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningRepresentation LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Two-Level Supervised Contrastive Learning for Response Selection in Multi-Turn Dialogue
Selecting an appropriate response from many candidates given the utterances in a multi-turn dialogue is the key problem for a retrieval-based dialogue system. Existing work formalizes the task as matching between the utt…
Contrastive LearningConversational Response SelectionRetrievalSentence+1Two-Level Supervised Contrastive Learning for Response Selection in Multi-Turn Dialogue
Selecting an appropriate response from many candidates given the utterances in a multi-turn dialogue is the key problem for a retrieval-based dialogue system. Existing work formalizes the task as matching between the ut…
Contrastive LearningRetrievalSentenceSTSRegion-level Contrastive and Consistency Learning for Semi-Supervised Semantic Segmentation
Current semi-supervised semantic segmentation methods mainly focus on designing pixel-level consistency and contrastive regularization. However, pixel-level regularization is sensitive to noise from pixels with incorrect…
SegmentationSemantic SegmentationSemi-Supervised Semantic SegmentationIs Self-Supervised Learning More Robust Than Supervised Learning?
Self-supervised contrastive learning is a powerful tool to learn visual representation without labels. Prior work has primarily focused on evaluating the recognition accuracy of various pre-training algorithms, but has o…
Contrastive LearningData AugmentationSelf-Supervised LearningCL-MVSNet: Unsupervised Multi-View Stereo with Dual-Level Contrastive Learning
Unsupervised Multi-View Stereo (MVS) methods have achieved promising progress recently. However, previous methods primarily depend on the photometric consistency assumption, which may suffer from two limitations: ind…
3D geometryContrastive LearningDepth Estimation