Improving embedding with contrastive fine-tuning on small datasets with expert-augmented scores
This paper presents an approach to improve text embedding models through contrastive fine-tuning on small datasets augmented with expert scores. It focuses on enhancing semantic textual similarity tasks and addressing text retrieval problems. The proposed method uses soft labels derived from expert-augmented scores to fine-tune embedding models, preserving their versatility and ensuring retrieval capability is improved. The paper evaluates the method using a Q\&A dataset from an online shopping website and eight expert models. Results show improved performance over a benchmark model across multiple metrics on various retrieval tasks from the massive text embedding benchmark (MTEB). The method is cost-effective and practical for real-world applications, especially when labeled data is scarce.
Code (0)
등록된 구현이 없습니다.
Tasks
RetrievalSemantic Textual SimilarityText RetrievalSimilar Papers 제목 키워드 기반
Improving Text Embeddings for Smaller Language Models Using Contrastive Fine-tuning
While Large Language Models show remarkable performance in natural language understanding, their resource-intensive nature makes them less accessible. In contrast, smaller language models such as MiniCPM offer more susta…
Language ModelingLanguage ModellingNatural Language UnderstandingSTSEmotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning
Contrastive learning techniques have been widely used in the field of computer vision as a means of augmenting datasets. In this paper, we extend the use of these contrastive learning embeddings to sentiment analysis tas…
Contrastive LearningSentiment AnalysisSemi-Supervised End-To-End Contrastive Learning For Time Series Classification
Time series classification is a critical task in various domains, such as finance, healthcare, and sensor data analysis. Unsupervised contrastive learning has garnered significant interest in learning effective represent…
ClassificationContrastive LearningTime SeriesTime Series Classification+1Benchmarking Image Embeddings for E-Commerce: Evaluating Off-the Shelf Foundation Models, Fine-Tuning Strategies and Practical Trade-offs
We benchmark foundation models image embeddings for classification and retrieval in e-Commerce, evaluating their suitability for real-world applications. Our study spans embeddings from pre-trained convolutional and tran…
BenchmarkingContrastive LearningTransfer LearningLM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning
In recent years, there has been significant progress in developing pre-trained language models for NLP. However, these models often struggle when fine-tuned on small datasets. To address this issue, researchers have prop…
Contrastive LearningData AugmentationLinguistic AcceptabilityNatural Language Inference+3