A simple method for domain adaptation of sentence embeddings
Pre-trained sentence embeddings have been shown to be very useful for a variety of NLP tasks. Due to the fact that training such embeddings requires a large amount of data, they are commonly trained on a variety of text data. An adaptation to specific domains could improve results in many cases, but such a finetuning is usually problem-dependent and poses the risk of over-adapting to the data used for adaptation. In this paper, we present a simple universal method for finetuning Google's Universal Sentence Encoder (USE) using a Siamese architecture. We demonstrate how to use this approach for a variety of data sets and present results on different data sets representing similar problems. The approach is also compared to traditional finetuning on these data sets. As a further advantage, the approach can be used for combining data sets with different annotations. We also present an embedding finetuned on all data sets in parallel.
Code (0)
등록된 구현이 없습니다.
Tasks
Domain AdaptationSentenceSentence EmbeddingsSimilar Papers 제목 키워드 기반
Efficient Domain Adaptation of Sentence Embeddings Using Adapters
Sentence embeddings enable us to capture the semantic similarity of short texts. Most sentence embedding models are trained for general semantic textual similarity tasks. Therefore, to use sentence embeddings in a partic…
Domain AdaptationSemantic SimilaritySemantic Textual SimilaritySentence+4Domain Adaptation for Japanese Sentence Embeddings with Contrastive Learning based on Synthetic Sentence Generation
Several backbone models pre-trained on general domain datasets can encode a sentence into a widely useful embedding. Such sentence embeddings can be further enhanced by domain adaptation that adapts a backbone model to a…
Contrastive LearningDomain AdaptationSemantic Textual SimilaritySentence+2Leveraging Auxiliary Tasks for Document-Level Cross-Domain Sentiment Classification
In this paper, we study domain adaptation with a state-of-the-art hierarchical neural network for document-level sentiment classification. We first design a new auxiliary task based on sentiment scores of domain-independ…
ClassificationDenoisingDomain AdaptationGeneral Classification+5Efficient Extraction of Pseudo-Parallel Sentences from Raw Monolingual Data Using Word Embeddings
We propose a new method for extracting pseudo-parallel sentences from a pair of large monolingual corpora, without relying on any document-level information. Our method first exploits word embeddings in order to efficien…
Domain AdaptationInformation RetrievalMachine TranslationSentence+2Incremental Domain Adaptation for Neural Machine Translation in Low-Resource Settings
We study the problem of incremental domain adaptation of a generic neural machine translation model with limited resources (e.g., budget and time) for human translations or model training. In this paper, we propose a nov…
Active LearningDomain AdaptationInformativenessMachine Translation+3