MetricBERT: Text Representation Learning via Self-Supervised Triplet Training
We present MetricBERT, a BERT-based model that learns to embed text under a well-defined similarity metric while simultaneously adhering to the ``traditional'' masked-language task. We focus on downstream tasks of learning similarities for recommendations where we show that MetricBERT outperforms state-of-the-art alternatives, sometimes by a substantial margin. We conduct extensive evaluations of our method and its different variants, showing that our training objective is highly beneficial over a traditional contrastive loss, a standard cosine similarity objective, and six other baselines. As an additional contribution, we publish a dataset of video games descriptions along with a test set of similarity annotations crafted by a domain expert.
Code (0)
등록된 구현이 없습니다.
Tasks
Representation LearningTripletMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
TriBYOL: Triplet BYOL for Self-Supervised Representation Learning
This paper proposes a novel self-supervised learning method for learning better representations with small batch sizes. Many self-supervised learning methods based on certain forms of the siamese network have emerged and…
Representation LearningSelf-Supervised LearningTripletSimTriplet: Simple Triplet Representation Learning with a Single GPU
Contrastive learning is a key technique of modern self-supervised learning. The broader accessibility of earlier approaches is hindered by the need of heavy computational resources (e.g., at least 8 GPUs or 32 TPU cores)…
Contrastive LearningGPUMedical Image AnalysisRepresentation Learning+2Leveraging triplet loss for unsupervised action segmentation
In this paper, we propose a novel fully unsupervised framework that learns action representations suitable for the action segmentation task from the single input video itself, without requiring any training data. Our met…
Action SegmentationClusteringMetric LearningSegmentation+3Training-free Zero-shot Composed Image Retrieval with Local Concept Reranking
Composed image retrieval attempts to retrieve an image of interest from gallery images through a composed query of a reference image and its corresponding modified text. It has recently attracted attention due to the col…
Image RetrievalRerankingRe-RankingRetrieval+2Trip-ROMA: Self-Supervised Learning with Triplets and Random Mappings
Contrastive self-supervised learning (SSL) methods, such as MoCo and SimCLR, have achieved great success in unsupervised visual representation learning. They rely on a large number of negative pairs and thus require eith…
Few-Shot LearningRepresentation LearningSelf-Supervised LearningTriplet+1